Behavior estimating method, behavior estimating system, service providing method, signal detecting method, signal detecting unit, and signal processing system

ABSTRACT

According to one embodiment, a behavior estimating method for estimating behavior of a person being measured equipped with an activity meter including an acceleration sensor, includes presetting a first angle between a gravitation direction and a direction of action of a standard person being measured, calculating different acceleration waveforms from the activity meter, extracting a second angle between the gravitation direction and the direction of action of the person from the acceleration waveforms, performing a coordinate transformation process among the acceleration waveforms based upon the present first angle and the extracted second angle, and performing behavior estimation of the person using a result of the coordinate transformation.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation Application of PCT Application No.PCT/JP2017/012758, filed Mar. 28, 2017 and based upon and claiming thebenefit of priority from Japanese Patent Application No. 2016-100007,filed May 18, 2016, the entire contents of all of which are incorporatedherein by reference.

FIELD

Embodiments described herein relate generally to a behavior estimationtechnology using a signal detected from an activity meter attached to aperson being measured.

However, the embodiments are not limited to the above technology, butinclude a detection technology of signals detected from a sensor, anestimation technology or a service providing technology using thedetection technology.

BACKGROUND

In activity estimation using an activity meter, when workers who arepersons being measured of the activity meter work using articles to beused outdoors, such as a device and equipment (e.g., dolly), workingestimation accuracy may vary with differences in physicalcharacteristics such as height among the workers.

Therefore, technologies to reduce variations in estimation accuracyamong workers are required.

Disturbance noise components mixed into signals detected from a numberof sensors as well as the activity meter also greatly affect detectionenvironment and detection condition; consequently, the accuracy ofdetection using a sensor varies with the detection environment anddetection condition.

On the other hand, individual optimization of a signal detecting method,which is performed for each detection environment or for each detectioncondition to reduce the variations in detection accuracy, has a harmfuleffect with significant complication.

Therefore, technologies capable of detecting a signal with smalldetection error by a simple method are required.

There is the conventional literature which discloses the technologyrelevant to the above-mentioned activity meter.

Conventionally, when workers moved dollies, correct waveform data of adetected signal used for activity estimation was prepared for each ofthe workers and activity estimation was done for each individual worker.It was therefore possible to reduce variations in estimation accuracyamong workers. To obtain correct waveform data for each of the workershowever needs labor and costs and greatly impairs convenience.

As a technology to remove disturbance noise components to be mixed intoa signal detected from a common sensor as well as behavior estimation ofworkers who move dollies as described above, the conventional literaturediscloses removing low-frequency components of a signal detected from anacceleration sensor using a high-pass filter. However, when a workerwishes to detect a slow change, the components of the detected signalwill be included in a low-frequency band. Therefore, this method incursthe risk of removing even the detected signal to the contrary.

When the acceleration sensor is attached to the worker's body, as amethod of estimating whether the worker's behavior is “walking state” or“running state”, the conventional literature discloses calculating theratio between “magnitude of amplitude” and “cycle” of a signal detectedfrom the acceleration sensor to discriminate between the walking andrunning states easily. However, even though such calculation isperformed, estimation accuracy is still low in the detection environmentwhere a number of disturbance noise components are mixed in the detectedsignal.

Therefore, even in the environment where disturbance noise componentsare mixed in the detected signal, it is desirable to ensure reliabilityof the signal detection.

BRIEF DESCRIPTION OF THE DRAWINGS

A general architecture that implements the various features of theembodiments will now be described with reference to the drawings. Thedrawings and the associated descriptions are provided to illustrate theembodiments and not to limit the scope of the invention.

FIG. 1 is an illustration of an example of the present embodiment.

FIG. 2 is an illustration (1) of a method for extracting a predeterminedcriterion from detected raw signals.

FIG. 3 is an illustration (2) of the method for extracting apredetermined criterion from detected raw signals.

FIG. 4 is an illustration of a coordinate transformation method among aplurality of detected signals.

FIG. 5 is an illustration of a situation where a person being measuredwho is smaller in stature than average.

FIG. 6 is an illustration of a situation where a person being measuredwho is taller in stature than average.

FIG. 7 is an illustration of variations in the kinetic friction forceand static friction force of a dolly which depend upon the direction ofaction.

FIG. 8 is an illustration of a person-being-measured behavior estimatingmethod.

FIG. 9 is an illustration of a coordinate transformation method among aplurality of detected signals in a three-dimensional direction.

FIG. 10 is an illustration of the relationship between the measuredacceleration waveform (detected raw signal) and the transformed standardmeasured acceleration in the three-dimensional direction.

FIG. 11 is an illustration of an example of application to a warehousemanagement system (WMS) of the present embodiment.

FIG. 12 is an illustration of an example of a signal processing systemin the present embodiment.

FIG. 13 is an illustration of a signal processing system according toanother embodiment.

FIG. 14 is an illustration of the relationship between criterion A(geomagnetism direction) and the coordinate axis along an activitymeter.

FIG. 15 is an illustration (1) of the relationship between criterion B(gravity direction) and the coordinate axis along an activity meter.

FIG. 16 is an illustration (2) of the relationship between criterion B(gravity direction) and the coordinate axis along the activity meter.

FIG. 17 is an illustration of the relationship between angle θyz of anarm of a person being measured and the detected raw signal waveform.

FIG. 18 is an illustration of the relationship between sample data andpattern matching target data.

FIG. 19 is an illustration of a pattern matching method in whichvariations in behavior are taken into consideration.

FIG. 20 is an illustration of an optimum route selection method.

FIG. 21 is an illustration of an example of hardware configuration in abehavior estimating system.

FIG. 22A is an illustration (1) of a processing flow in the systemaccording to the present embodiment.

FIG. 22B is an illustration (2) of the processing flow in the systemaccording to the present embodiment.

FIG. 22C is an illustration (3) of the processing flow in the systemaccording to the present embodiment.

FIG. 23 is an illustration of how to form an angle different from thatin FIG. 15 .

FIG. 24 is an illustration of how to form an angle different from thatin FIG. 2 .

FIG. 25 is an illustration of a signal processing method in which aworker carries a thing with hands.

FIG. 26 is an illustration of an application example using detectedsignals from different sensors.

FIG. 27 is an illustration of an example of a method for processingdetected raw signals from an elevator moving up and down.

FIG. 28 is an illustration of an application example in which anacceleration sensor and an angular velocity sensor are combined.

FIG. 29A is an illustration (1) of another embodiment of the criterionextraction method.

FIG. 29B is an illustration (2) of another embodiment of the criterionextraction method.

FIG. 29C is an illustration (3) of another embodiment of the criterionextraction method.

FIG. 30 is an illustration of an application example using apredetermined reference regarding a photoelectric signal.

FIG. 31 is an illustration of an example of a processing flow regardingan application example using a photoelectric signal.

FIG. 32 is an illustration of an application example using apredetermined reference regarding an audio signal.

FIG. 33 is an illustration of frequency spectral characteristics of ageneral audio signal.

FIG. 34 is an illustration of an example of a method for extracting acriterion from audio signals.

FIG. 35 is an illustration of a tendency to the contents of processingin the present embodiment.

FIG. 36A is an illustration (1) of a method for mixing a disturbancesignal into the extracted criterion.

FIG. 36B is an illustration (2) of the method for mixing a disturbancesignal into the extracted criterion.

FIG. 36C is an illustration (3) of the method for mixing a disturbancesignal into the extracted criterion.

FIG. 37 is an illustration of a tendency to an extension/contractionmatching process in the present embodiment.

FIG. 38 is an illustration of a configuration of a glasses-type wearableterminal (drive device).

FIG. 39A is a flowchart of an operation example of an applicationexample of the system according to the present embodiment.

FIG. 39B is a flowchart of another operation example of the applicationexample of the system according to the present embodiment.

FIG. 40 is an illustration of an application example of the presentembodiment.

FIG. 41 is an illustration of a detailed configuration of a sensordevice that detects and notifies a work completion state.

FIG. 42 is an illustration of a basic configuration of the interior ofan environmental vibration power-generating device.

FIG. 43 is an illustration (1) of the principle of storage of theinterior of the environmental vibration power-generating device.

FIG. 44 is an illustration (2) of the principle of storage of theinterior of the environmental vibration power-generating device.

FIG. 45 is an illustration (3) of the principle of storage of theinterior of the environmental vibration power-generating device.

FIG. 46 is an illustration (4) of the principle of storage of theinterior of the environmental vibration power-generating device.

FIG. 47 is an illustration (5) of the principle of storage of theinterior of the environmental vibration power-generating device.

FIG. 48 is an illustration of another embodiment of a configuration ofpart of the interior of the sensor device.

FIG. 49 is an illustration (1) of arrangement of a plurality ofinstantaneous voltage generators in the sensor device.

FIG. 50 is an illustration (2) of arrangement of a plurality ofinstantaneous voltage generators in the sensor device.

FIG. 51 is an illustration of a method for detecting a value of thechanged acceleration/angular velocity in the controller.

FIG. 52 is an illustration of vibration characteristics before and aftera screwing operation.

FIG. 53 is an illustration of a changed angular velocity state when adoor is closed.

DETAILED DESCRIPTION

In the present embodiment, the coordinate system of measuredacceleration is transformed into a coordinate system capable ofestimating behavior of a worker (person being measured) using an anglebetween the arm of the worker and a dolly, extracted from the resolvedgravity acceleration value. The behavior of the worker is estimatedusing the transformed detected signal. The behavior estimation isimproved in accuracy by automatically correcting the conditions of thebehavior estimation.

According to the present embodiment, coordinate transformation isperformed for a plurality of detected signal waveforms using an anglebetween a reference direction such as a gravitation direction and adirection of action of the person being measured and based upon theresult, the behavior of the person being measured is estimated orservice is provided to the person being measured.

The direction of action can be adapted to a direction in which theperson being measured pushes the dolly or an angle of the arms of theperson being measured who holds the dolly.

As a result of the present embodiment, the accuracy of behaviorestimation can be increasing without preparing or using correct waveformdata corresponding to each worker whose behavior is to be estimated.

Various embodiments will be described hereinafter with reference to theaccompanying drawings. In this specification, a sensor device attachableto the body of a person and capable of detecting an amount of activityof the person using a signal detected therefrom, will be referred to asan activity meter. As an example of a method for attaching the activitymeter to the human body (person being measured 2), an activity meter(sensor device) 4 is shaped like a wristband and attached to the arm orleg of the person being measured 2 as shown in (a) of FIG. 1 . In theexample shown in (a) of FIG. 1 , the person being measured 2 equippedwith the activity meter 4 pushes a dolly 122, and behavior estimation asto whether “the person being measured 2 is moving 22 the dolly 122” ismade from raw signals detected and collected from the activity meter 4.The activity meter is not limited to the wristband, but can be embeddedin clothes and clothing such as hats, glasses and shoes. The basiccontents of the present embodiment are to perform the following steps(1) to (6) (some of the steps can be omitted according to thecircumstances):

-   -   (1) Detecting raw signals from the activity meter attached to        the human body (or animal);    -   (2) Extracting a criterion from the detected raw signals;    -   (3) Processing the detected raw signals based on the extracted        criterion;    -   (4) Estimating the behavior (or condition or demand) of the        human body (or animal) based on a result of the signal        processing;    -   (5) Analyzing the history of behavior for each person being        measured (worker) (who is in business, working, etc.); and    -   (6) Improving business (improving a working process) based on a        result of the analysis.

The embodiment is not limited to the above steps, but may include a stepof (7) providing proper service using a result of the estimationobtained from steps (1) to (4). Furthermore, the system of the presentembodiment is not limited to this step, but may include any applicationusing the above result of the estimation.

Moreover, the present embodiment may include steps of fixing a sensordevice to a given object (other than persons and animals), estimating acondition related to the given object based upon a signal detected fromthe sensor device and providing service based upon the result.

In FIG. 1 , (b) shows variations in behavior of the person beingmeasured 2 every elapsed time and variations in raw signals(acceleration waveforms) detected from the activity meter 4 during eachbehavior. Specifically, the person being measured 2:

-   -   performs manual operation 20 from 9:00 to 9:01;    -   moves the dolly from 9:01 to 9:03;    -   stops temporarily until 9:04; and    -   walks from 9:04 to 9:06.

In the above behavior, as shown in (b) of FIG. 1 , a raw signal(acceleration waveform) proper (unique) to each behavior is detectedfrom the activity meter 4.

Therefore, the behavior of the person being measured 2 can be estimatedby extracting characteristics of the detected raw signals (accelerationwaveforms) using a behavior estimation engine 56. In the presentembodiment, the raw signals (acceleration waveforms) detected from theactivity meter 4 are stored temporarily in a memory unit (outputwaveform data storage that stores waveform data output from a sensor)(corresponding to the above step (1)), and a series of steps of (2)criterion extraction, (3) signal processing, (4) behavior estimation,(5) history analysis and (6) business improvement 10 is performed usingthe detected raw signals (acceleration waveforms) read out of the memoryunit (output waveform data storage unit).

The present embodiment is not limited to the above steps. The steps (2)to (6) can be performed in real time for the raw signals (accelerationwaveforms) detected from the activity meter 4 in step (1). Furthermore,based upon a behavior estimation result 16, the step (7) (of providingproper service) can be performed.

As will be described later with reference to FIG. 3 , three axes (X, Yand Z axes) which are orthogonal to one another are preset in theinterior of a signal detection unit 60 of the activity meter 4 and asignal detection unit 60 of a sensor device 6. As the relationshipbetween these axes and a predetermined criterion (e.g., a gravitydirection along the z axis and a geomagnetism direction along the yaxis), the direction from the arm 120 of the person being measured 2equipped with the activity meter 4 toward the middle finger thereof isset as the Y axis, as will be described later with reference to FIG. 5 ,for example. Here, the action direction 100 in which the person beingmeasured 2 pushes the dolly (the direction in which force is exerted bythe person being measured 2) coincides with the Y-axis direction.

The axial direction perpendicular to the back of a hand of the personbeing measured is set as the Z-axis direction and the directionorthogonal to the Y and Z axes (direction from the little finger of theperson being measured 2 toward the thumb thereof) is set as the X axis.In accordance with these coordinate axes, a three-axis accelerationsensor 72 outputs acceleration waveforms (detected raw signals WX(t),WY(t) and WZ(t)) in the X-axis direction, Y-axis direction and Z-axisdirection.

At the same time, a three-axis geomagnetism sensor 74 outputs fieldintensity signals in the X-axis, Y-axis and Z-axis directions.

On the other hand, the direction along criterion A (geomagnetismdirection) 102 is defined as the y axis (direction from the South Pole114 to the North Pole 112). The direction from the west to the east isalso defined as the x axis.

In most cases, the Y and y axes do not coincide with each other, and thewaveforms of Wx(t) and Wy(t) are not directly detected as raw signalsbecause the Y and y axes are inclined to each other. Based on thesepremises, the following explanation can be given.

The three-axis acceleration sensor 72 (FIG. 17 ) embedded in theactivity meter 4 detects raw signals WX(t), WY(t) and WZ(t). The rawsignal WZ(t)(or WY(t)) corresponds to activity meter data (accelerationwaveform) 12 shown in FIG. 1 .

The detected raw signals WX(t), WY(t) and WZ(t) collected sequentiallyas time elapses are stored sequentially in a memory unit (outputwaveform data storage unit) 82 shown in FIG. 17 . The detected rawsignals WX(t), WY(t) and WZ(t) stored at once in the memory unit (outputwaveform data storage unit) 82 may data corresponding to continuousbehavior of the person being measured 2 for two to three hours. Not onlythis data but also behavior data of the person being measured 2 for oneday can be stored at once.

FIG. 2 shows a method for extracting criterion B (geomagnetismdirection) 104 using signal components Q41L and Q42L that have beensubjected to a low-pass filtering process, which represent acceleration.There is the following relationship among low-frequency component Q42Lof the raw signal WY(t) detected in the Y-axis direction, low-frequencycomponent Q41L of the raw signal WZ(t) detected in the Z-axis directionand gravity acceleration component G in the z-axis direction.θyz=arctan(Q42L/Q41L)  Equation (1)θyz=arcsin(Q42L/G)  Equation (2)θyz=arccos(Q41L/G)  Equation (3)

Using one of the above equations (1), (2) and (3), angle θyz can becalculated.

The function of reference direction extraction 92 and the function ofangle extraction 94 shown in FIGS. 12 and 13 respectively correspond toa function of extracting the criterion B (gravity direction) 104 fromthe detected raw signals to calculate angle θ in FIGS. 3 and 2 and afunction of extracting the criterion A (geomagnetism direction) 102 fromthe detected raw signals to calculate angle θ in FIGS. 2 and 3 .

Furthermore, the function of detected signal generation (signalprocessing) whose noise components have been reduced shown in FIGS. 12and 13 corresponds to a function of processing the detected raw signalsWX(t), WY(t) and WZ(t) using equations (4) to (7) and converting theprocessed signals into detected signals “WY(t)|θyz=θs” and Wy(t) in apredetermined direction.

When the arm 120 of the person being measured 2 equipped with theactivity meter 4 is vibrated, the vibration is mixed into the detectedraw signals WX(t), WY(t) and WZ(t) as disturbance nose components. Inmost cases, the frequency due to the vibration of the arm 120 of theperson being measured 2 is 0.5 Hz or higher.

Therefore, the foregoing disturbance noise components can be reduced byextracting the signal components Q41L, Q42L and Q43L of the detected rawsignals WZ(t), WY(t) and WX(t) which has been subjected to a low-passfiltering process 138, as shown in FIG. 3 . When the characteristics ofgain g with respect to the frequency f of the low-pass filter arecharacteristics shown in FIG. 3 , only signal components whose frequencyis cutoff frequency fc or lower can be extracted. The value of cutofffrequency fc when a low-pass filtering process 130 is performed can beset to 60 Hz or 50 Hz from the above description and may be set to 0.5Hz.

The present embodiment is not limited to the frequency characteristicshown in FIG. 3 , but may adopt all filtering processes. For example, aband-pass filtering process to extract specific frequency componentsonly can be performed.

If any signal processing is performed for the detected raw signalsWZ(t), WY(t) and WX(t) to reduce disturbance noise components asdescribed above, the advantage in which a predetermined criterion(direction) can be extracted with high accuracy is brought about.

The following is a description of a method for transforming a coordinatesystem of measured acceleration into a coordinate system capable ofestimating behavior of the person being measured 2 using an anglebetween the arm of the person being measured 2 and the dolly 122,extracted from the resolved gravity acceleration value, in order toreduce variations in working estimation accuracy due to differences inphysical characteristics such as height among workers.

FIG. 4 shows a coordinate transformation state between y axis/z axis andY axis/Z axis when the X axis and x axis coincide with each other. Asshown in FIG. 4 , the transformation equation between waveformWy(t)/Wz(t) and raw signal WY(t)/WZ(t) detected after the transformationis expressed as follows.

$\begin{matrix}\begin{matrix}{\begin{Bmatrix}{{WX}(t)} \\{{WY}(t)} \\{{WZ}(t)}\end{Bmatrix} = \begin{Bmatrix}{{Wx}(t)} \\{{\cos\;\theta\;{{yzWy}(t)}} + {\sin\;\theta\;{{yzWz}(t)}}} \\{{{- \sin}\;\theta\;{{yzWy}(t)}} + {\cos\;\theta\;{{yzWz}(t)}}}\end{Bmatrix}} \\{= {\begin{Bmatrix}1 & 0 & 0 \\0 & {\cos\;\theta\;{yz}} & {\sin\;\theta\;{yz}} \\0 & {{- \sin}\;\theta\;{yz}} & {\cos\;\theta\;{yz}}\end{Bmatrix}\begin{Bmatrix}{{Wx}(t)} \\{{Wy}(t)} \\{{Wz}(t)}\end{Bmatrix}}}\end{matrix} & {{Equation}\mspace{14mu}(4)} \\{{{the}\mspace{14mu} X\mspace{14mu}{axis}\mspace{14mu}{and}\mspace{14mu} x\mspace{14mu}{a{xis}}\mspace{14mu}{coincide}\mspace{14mu}{with}\mspace{14mu}{each}\mspace{14mu}{other}}\;} & \;\end{matrix}$

The use of equation (4) thus makes it possible to calculate signalsWx(t), Wy(t) and Wz(t) detected after the transformation, which could beobtained in the directions of the y axis parallel to the criterion A(geomagnetism direction) 102 and the z axis (and x axis orthogonalthereto) parallel to the criterion B (gravity) 104 from the raw signalsWX(t), WY(t) and WZ(t) detected from the three-axis acceleration sensor72 (or three-axis geomagnetism sensor 74).

Then, when X axis and Xs axis coincide with each other, an angle betweenY axis and Ys axis and an angle between Z axis and Zs axis are definedby a difference value “dθyz=θs−θyz” therebetween.

Using the relationship in rotating coordinate transformation between Yand Z axes and Ys and Zs axes, the transformation equation to transformthe measured acceleration waveforms (detected raw signals) WY(t) andWZ(t) into standard measured acceleration WYs(t) and WZs(t) is given asfollows.

$\begin{matrix}\begin{matrix}{\begin{Bmatrix}{{WXs}(t)} \\{{WYs}(t)} \\{{WZs}(t)}\end{Bmatrix} = \begin{Bmatrix}{{WXs}(t)} \\{{\cos\; d\;\theta\;{{yzWYs}(t)}} + {\sin\; d\;\theta\;{{yzWZs}(t)}}} \\{{{- \sin}\; d\;\theta\;{{yzWYs}(t)}} + {\cos\; d\;\theta\;{{yzWZs}(t)}}}\end{Bmatrix}} \\{= {\begin{Bmatrix}1 & 0 & 0 \\0 & {\cos\; d\;\theta\;{yz}} & {\sin\; d\;\theta\;{yz}} \\0 & {{- \sin}\; d\;\theta\;{yz}} & {\cos\; d\;\theta\;{yz}}\end{Bmatrix}\begin{Bmatrix}{{WX}(t)} \\{{WY}(t)} \\{{WZ}(t)}\end{Bmatrix}}}\end{matrix} & {{Equation}\mspace{14mu}(5)} \\{{the}\mspace{14mu} X\mspace{14mu}{axis}\mspace{14mu}{and}\mspace{14mu} x\mspace{14mu}{axis}\mspace{14mu}{coincide}\mspace{14mu}{with}\mspace{14mu}{each}\mspace{14mu}{other}} & \;\end{matrix}$

When the three-axis acceleration sensor 72 (or three-axis geomagnetismsensor 74) is used as a sensor, a plurality of detected raw signals thatvary as time elapses, such as Wx(t), Wy(t) and Wz(t), are collected fromone sensor or a plurality of sensors. If the detected raw signals areprocessed (as a result of processing of, e.g., reducing noise componentson the Wy(t) side instead of collecting noise components on the Wz(t)side as a result of uneven distribution of noise components) usingcharacteristics of the detected raw signals (extraction of predeterminedoriginal information such as θyz contained therein), the advantage thatthe noise components of a given detected signal (e.g., Wy(t)) that hasbeen processed can be reduced, is brought about. The present embodimentis not limited to the noise reduction. The detected raw signals can beprocessed to extract a specific signal component from the detected rawsignals (e.g., extraction of gravity acceleration value G using equation(1)).

As one example of extracting predetermined original information from thedetected raw signals, an example of extracting a predetermined criterion(direction) using equations (1) to (3) by extracting predeterminedfrequency components only using the low-pass filtering process 138 shownin FIG. 3 has already been described. However, the present embodiment isnot limited to this example but may include an example of extracting“all types of information” contained in the detected raw signals. Forexample, noise components (rolling components 208) contained in both thedetected raw signals WY(t) and WZ(t) can be caused to correspond to“predetermined information contained in the detected raw signals”, aswill be described with reference to FIG. 26 .

The detected raw signals collected simultaneously from sensors ofdifferent types such as the three-axis acceleration sensor 72 and athree-axis angular velocity sensor 212 can be processed. For example,“predetermined original information contained in the detected rawsignals” in the embodiment shown in FIG. 28 (the details thereof will bedescribed later) corresponds to “rotation direction 218 of steeringwheel 214”, “acceleration/deceleration direction of a bus, a truck or anautomobile 210”, “the change in moving direction of a bus, a truck or anautomobile 210” or the like.

As a method for extracting these original information, for example, aspecific sensor attachment direction (the direction of a predeterminedcoordinate axis set in a specific sensor) can be distinguished and aspecific signal component (e.g., detection of signal characteristicssuch as an absolute value of detected acceleration increasing at thestart of acceleration/deceleration and converging to zero as the speedapproaches a constant speed) can be used.

The following is a description of how to handle the dolly 122 for aperson being measured 140 of average height, a person being measured 142who is smaller than average or a person being measured 144 who is tallerthan average. As shown in FIG. 5 , angle θyz formed when the personbeing measured 142 pushes the dolly 122 is smaller than angle θs formedwhen the person being measured 140 pushes the dolly 122 (θyz<θs). On theother hand, as shown in FIG. 6 , angle θyz formed when the person beingmeasured 144 pushes the dolly 122 is larger than angle θs (θyz>θs).

As an application example of the present embodiment described withreference to FIGS. 5 and 6 , a feedback method (a kind of learningfunction) regarding a threshold value used in a process of estimatingbehavior of the person being measured 142 who is smaller than average orthe person being measured 144 who is taller than average, will bedescribed. When the person being measured 144 who is taller than averagepushes the dolly 122 ((a) in FIG. 7 ), the value of angle θyz formedbetween the direction of action 100 and the direction (y axis) in whichthe dolly 122 moves increases. In this case, the force by which theperson being measured 144 presses the dolly 122 on the floor surfaceincreases. Accordingly, the kinetic friction force and static frictionforce of the dolly 122 increase and the dolly 122 becomes difficult tomove.

When the person being measured 142 who is smaller than average pushesthe dolly 122 ((b) in FIG. 7 ), the value of angle θyz formed betweenthe direction of action 100 and the direction (y-axis direction) inwhich the dolly 122 moves decreases. Thus, the force by which the personbeing measured 142 presses the dolly 122 on the floor surface becomesrelatively weak. Accordingly, the kinetic friction force and staticfriction force of the dolly become weak and the dolly 122 becomes easyto move. The variations in kinetic friction force and static frictionforce therefore affect the behavior estimation of the person beingmeasured 2.

Here, angle θs formed between the direction of action 100 in which theperson being measured 140 of average height pushes the dolly 122 and thedirection (y-axis direction) in which the dolly 122 moves is consideredto be a criterion. FIG. 8 shows the relationship in Euclidian distancebetween the transformed standard measured acceleration with respect toangle θyz formed when the person being measured 2 pushes the dolly 122based on angle θs and the amount of error among the measuredacceleration waveforms (detected raw signals) or the standard measuredacceleration waveforms (sample data).

In θyz≈θs, the characteristics shown in FIG. 8 are represented by avalue that is considerably smaller than a determination threshold value150 during behavior estimation of a person being measured. However, itis understood that as angle θyz greatly deviates from θs, thecharacteristics becomes larger than the determination threshold value150 during behavior estimation of a person being measured and theaccuracy of the behavior estimation lowers. To prevent the accuracy fromlowering, in the present embodiment, when angle θyz greatly deviatesfrom θs, the determination threshold value 150 during the behaviorestimation is automatically corrected (the determination threshold value150 is largely reset). This brings about the advantage that the behaviorof the person being measured 2 can be estimated with high accuracy eventhough angle θyz greatly deviates from θs.

The above-described equation (4) is a transformation equation in thecase where the X axis and x axis coincide with each other. In otherwords, FIG. 4 based on which the above equation (4) is derived, showsthat the y axis and z axis are rotated by θyz and transformed into the Yaxis and Z axis with the X axis and x axis coinciding with each other(coordinate transformation of vertical rotation⋅⋅3D transformation)

In contrast, (a) in FIG. 9 shows that the z axis and x axis are rotatedby θzx and transformed into the Z axis and X axis with the Y axis and yaxis coinciding with each other (coordinate transformation of horizontalrotation⋅⋅rotation transformation). Similarly, (b) in FIG. 9 shows thatthe x axis and y axis are rotated by θxy and transformed into the X axisand Y axis with the Z axis and z axis coinciding with each other.

Like the equation (4) derived based on FIG. 4 , the following generaltransformation equation is derived using FIG. 4 and (a) and (b) of FIG.9 .

$\begin{matrix}\begin{matrix}{\begin{Bmatrix}{{WX}(t)} \\{{WY}(t)} \\{{WZ}(t)}\end{Bmatrix} = \begin{Bmatrix}{\cos\;\theta\;{xz}} & {\sin\;\theta\;{xy}} & 0 \\{{- \sin}\;\theta\;{xy}} & {\cos\;\theta\;{xy}} & 0 \\0 & 0 & 1\end{Bmatrix}} \\{\begin{Bmatrix}{\cos\;\theta\;{zx}} & 0 & {{- \sin}\;\theta\;{zx}} \\0 & 1 & 0 \\{\sin\;\theta\;{zx}} & 0 & {\cos\;\theta\;{zx}}\end{Bmatrix}} \\{\begin{Bmatrix}1 & 0 & 0 \\0 & {\cos\;\theta\;{yz}} & {\sin\;\theta\;{yz}} \\0 & {{- \sin}\;\theta\;{yz}} & {\cos\;\theta\;{yz}}\end{Bmatrix}\begin{Bmatrix}{{Wx}(t)} \\{{Wy}(t)} \\{{Wz}(t)}\end{Bmatrix}}\end{matrix} & {{Equation}\mspace{14mu}(6)} \\{{general}\mspace{14mu}{transformation}\mspace{14mu}{equation}} & \;\end{matrix}$

A signal processing method to reduce the influence of irregularities ofa floor surface 128 will be described in the case where accelerationvariation components corresponding to the irregularities of the floorsurface 128 to be superimposed on the signal component Wz(t) areconsidered to be noise components as shown in, for example, (b) of FIG.17 described below. The noise components are mixed into a detected rawsignal “WY(t)|θyz<θs” shown in (c) of FIG. 17 and a detected raw signal“WY(t)|θyz>θs” shown in (e) of FIG. 17 . Consider a case where adetected signal of Wy(t) with few noise components can be obtained byperforming signal processing using the detected raw signal“WY(t)|θyz<θs” and detected raw signal “WY(t)|θyz>θs”. The above signalprocessing means extracting the detected signal of Wy(t) obtained byreducing noise components from the detected raw signal “WY(t)|θyz<θs” or“WY(t)|θyz>θs” into which a large number of noise components are mixed.This corresponds to the function of the detected signal extraction(signal processing) 96 after reduction of noise components in FIG. 13 .

Then, as a signal processing method to reduce the influence of height ofthe person being measured 2, a method for transforming the detected rawsignals (“measured acceleration waveforms” described later, namely,detected raw signals) WX(t), WY(t) and WZ(t) shown in (c) or (e) of FIG.17 into detected signals (transformed standard measured accelerationdescribed later) “WY(t)|θyz=θs” (or WYs(t)) when θyz=θs, correspondingto the person being measured 140 of average height, will be described.

Here, as shown in (a) of FIG. 10 , the angle formed between the Z axisand Zs axis and the angle formed between the X axis and Xs axis when theY axis and Ys axis coincide with each other are represented by dθzx.Similarly, as shown in (b) of FIG. 10 , the angle formed between the Xaxis and Xs axis and the angle formed between the Y axis and Ys axiswhen the Z axis and Zs axis coincide with each other are represented bydθxy.

The following general transformation equation to transform the measuredacceleration waveforms (detected raw signals) WX(t), WY(t) and WZ(t)when the X axis and Xs axis do not coincide with each other into thestandard measured accelerations WXs(t), WYs(t) and WZs(t) is given byexpanding the above equation (5) from FIG. 4 and (a) and (b) of FIG. 10.

$\begin{matrix}\begin{matrix}{\begin{Bmatrix}{{WXs}(t)} \\{{WYs}(t)} \\{{WZs}(t)}\end{Bmatrix} = \begin{Bmatrix}{\cos\; d\;\theta\;{xz}} & {\sin\; d\;\theta\;{xy}} & 0 \\{{- \sin}\; d\;\theta\;{xy}} & {\cos\; d\;\theta\;{xy}} & 0 \\0 & 0 & 1\end{Bmatrix}} \\{\begin{Bmatrix}{\cos\; d\;\theta\;{zx}} & 0 & {{- \sin}\; d\;\theta\;{zx}} \\0 & 1 & 0 \\{\sin\; d\;\theta\;{zx}} & 0 & {\cos\; d\;\theta\;{zx}}\end{Bmatrix}} \\{\begin{Bmatrix}1 & 0 & 0 \\0 & {\cos\; d\;\theta\;{yz}} & {\sin\; d\;\theta\;{yz}} \\0 & {{- \sin}\; d\;\theta\;{yz}} & {\cos\; d\;\theta\;{yz}}\end{Bmatrix}\begin{Bmatrix}{{WX}(t)} \\{{WY}(t)} \\{{WZ}(t)}\end{Bmatrix}}\end{matrix} & {{Equation}\mspace{14mu}(7)} \\{{general}\mspace{14mu}{transformation}\mspace{14mu}{equation}} & \;\end{matrix}$

The measured acceleration waveforms (detected raw signals) shown in (c)or (e) of FIG. 17 can thus be transformed into the standard measuredacceleration “WY(t)|θyz=θs” (or Ws) corresponding to θyz=θs.

As described above, the measured acceleration waveforms (detected rawsignals) WX(t), WY(t) and WZ(t) obtained from the activity meter 4attached to the person being measured 142 who is smaller than averageand the person being measured 144 who is taller than average aretransformed into WXs(t), WYs(t) and WZs(t) (standard measuredacceleration transformed into a corresponding average height). Basedupon the transformed detected signals (standard measured acceleration),behavior of the persons being measured 142 and 144 can be estimated.This brings about the advantage of allowing behavior of the personsbeing measured 140 to 144 to be estimated with high accuracy on a commonscale regardless of a difference in height among the persons beingmeasured 140 to 144.

A method of using a rotating coordinate transformation between thecoordinate axes shown in FIG. 4 has been so far described as a method ofdetecting a signal to reduce the influence of height of the person beingmeasured 2 by signal processing. However, the present embodiment is notlimited to this method but may use any other signal processing methodsif they use the raw signals Wx(t), Wy(t) and Wz(t) detected from thedifferent sensors 72 and 74. As one example, the detected raw signalsWx(t), Wy(t) and Wz(t) can be regarded as a three-dimensional vector toperform vector operation among vector components (e.g., vectorsynthesis).

The present embodiment is not limited to the above signal processingmethods. The raw signals detected from the different sensors 72 and 74can be processed to perform detected signal generation (signalprocessing) 96 (FIG. 12 or 13 ) after reduction of noise components. Inother words, in the conversion between the detected signals using theequation (4) calculated with reference to FIG. 4 , conversion betweensignals detected from, e.g., the three-axis acceleration sensor 72 onlyis performed. However, the present embodiment is not limited to thisconversion. For example, signal processing can be performed by combiningsignals detected from the three-axis geomagnetism sensor 74 with thesignals detected from the sensor 72 to reduce noise components of thedetected raw signals.

An example of behavior estimation of the person being measured 2 using adetected signal after the rotation transformation (signal processing)will be described below.

Angle θyz of the arm 120 of the person being measured 2 who is pushingthe dolly 122 changes over time. Thus, the process of calculating thedetected signal after rotation transformation (signal processing) needsto be repeated many times.

Prior to the description of an example of behavior estimation performedby a behavior estimating system 52 shown in FIG. 21 (or FIG. 12 or 13 ),the terms to be used here will be defined. One or more of the detectedraw signals WX(t), WY(t) and WZ(t) collected from the three-axisacceleration sensor 72 will be referred to as “measured accelerationwaveforms (detected raw signals)”. The signals detected when θyz=θs (oneor more thereof) will be referred to as “standard acceleration”. Thedetected raw signals WX(t)|θyz=θs, WY(t)|θyz=θs, and WZ(t)|θyz=θs (oneor more thereof) obtained directly when the person being measured 140 ofaverage height (FIG. 5 or 6 ) pushes the dolly 122 and collected fromthe three-axis acceleration sensor 72, corresponds to the “standardacceleration” described above.

When the above standard acceleration (detected raw signal) is stored inthe memory unit 82 (FIG. 12 ) and used as reference data (sample data)during the behavior estimation, it will be referred to as “standardmeasured acceleration waveform (sample data)”. The above standardmeasured acceleration waveform (sample data) may be sample datacollected only once from the only person being measured 140 of averageheight, an average value of data collected more than once, or an averagevalue of data obtained from a plurality of persons being measured 140 ofaverage height.

As has been described with reference to FIGS. 5 and 6 , the detectedsignals obtained after the standard acceleration transformed using theequation (5) or (7) based on the detected raw signals collected from theperson being measured 142 who is smaller than average or the personbeing measured 144 who is taller than average, will be referred to as“standard measured acceleration after transformation”.

In the foregoing process, a process “from collection of the detected rawsignals to signal processing (rotation transformation of the detectedsignals)” will be referred to as “cycle”. The number of times the cycleis repeated for one second will be referred to as “cycle frequency” andthe time required for one cycle will be referred to as “cycle period”.The shorter the cycle period, the more finely and accurately informationabout the behavior of the person being measured 2 can be collected.Thus, the cycle period corresponds to the “resolution” (informationcollecting speed) of the cycle processing.

In the present embodiment, statistical processing is performed afterdata of the transformed standard measured acceleration obtained for eachcycle is stored for a predetermined period. The storage period of thedata of the transformed standard measured acceleration necessary for thestatistical processing will be therefore referred to as “statisticalprocessing storage period”.

Furthermore, the results of statistical processing calculated for eachstatistical processing storage period are collected for a predeterminedperiod to estimate the behavior of the person being measured 2. The unitperiod (period during which the results of statistical processing arecollected) of the behavior estimation will be referred to as “behaviorestimation determination period”.

In the system (behavior estimating system 52) of the present embodiment,the higher the cycle frequency, the higher the resolution of the cycleprocessing, which facilitates detection of a rapid change in behavior ofthe person being measured 2. In the example of (b) of FIG. 1 , thebehavior (working) of the person being measured 2 changes every minuteor every two minutes from the manual operation 20 to dolly movement 20,stopping, and walking 26. When the person being measured 2 is busy, hisor her behavior (working) may change more frequently. When the personbeing measured 2 is busy, if the cycle of collection/signal processingof the detected raw signals is once per minute, the behavior (working)of the person being measured 2 which changes frequently cannot be finelypursued. The cycle period in the present embodiment is therefore set tonot longer than 10 seconds at most.

When the person being measured 2 is very busy, his or her behavior(working) may change in seconds. To deal with such a rapid change, it isdesirable that the cycle period be not longer than 0.5 seconds in thepresent embodiment. In the following description, 20 Hz is provisionallyset as an example of the cycle frequency.

An example has been described with reference to FIG. 3 , in which thevalue of cutoff frequency fc is set to 50 Hz or 60 Hz when apredetermined criterion is extracted. The value of cutoff frequency fcis somewhat consistent with the foregoing cycle frequency of 20 Hz. If,however, the cutoff frequency fc is set to a low value of 0.5 Hz or thelike, it is technically impossible to subject the detected raw signalsto the low-pass filtering process 138 for each cycle. In this case, thelow-pass filtering process 138 can be performed using the detected rawsignals in a plurality of cycles.

As has been described, in the system (signal processing system) of thepresent embodiment, the raw signals detected from the three-axisacceleration sensor 72 and the three-axis geomagnetism sensor 74(digital signals obtained by an AD converter 78 through a signalamplifier group 76) are stored temporarily in the memory unit 82, thenread out in response to an instruction from a control unit 80 andprocessed. Therefore, the low-pass filtering process 138 using thedetected raw signals between a plurality of cycles is facilitated.However, the present embodiment is not limited to this signalprocessing. If the signals are properly processed in the memory unit 82,the signal processing can be performed in real time.

The following is a description of the statistical processing storageperiod described above. For example, statistical processing for data foronly one cycle is meaningless. To perform effective statisticalprocessing, it is desirable to store data for at least four or morecycles, preferably ten or more cycles. For example, when the cyclefrequency is 20 Hz (cycle period is 50 mS), 0.5 seconds are required asthe statistical processing storage period in order to store data for tencycles.

If the above determination period for behavior estimation is not shorterthan the statistical processing storage period as a minimum requirement,the behavior of the person being measured 2 can be estimated. Toestimate the behavior more accurately, however, the determination periodof the behavior estimation needs to be four or more times longer thanthe statistical processing storage period (preferably eight or moretimes). For the above reason, four seconds can be set as thedetermination period of the behavior estimation.

From the above descriptions, the relationship among the periods is asexpressed by “determination period of behavior estimation”≥“statisticalprocessing storage period”≥“cycle period”. If the behavior of the personbeing measured 2 is estimated for each determination period of behaviorestimation (e.g., every four seconds), service providing to the personbeing measured 2 is greatly delayed. Therefore, in the system (behaviorestimating system 52) according to the present embodiment, the behaviorestimation process is continued while shifting every cycle period.Similarly, the statistical processing storage period is sequentially setwhile shifting every cycle period.

As a method for estimating activity of the person being measured 2 usingthe detected signals (standard measured acceleration) which have beenprocessed, one of the following determinations or a combination thereofis made in the present embodiment.

-   -   (A) Determination of irregularities and dispersion among the        detected signals before and after rotation transformation        (signal processing);    -   (B) Determination of similarities between the transformed        standard measured acceleration and sample data; and    -   (C) Determination of expansion/contraction matching between the        transformed standard measured acceleration and sample data.

The above behavior estimation methods will be described.

-   -   (A) Determination of irregularities and dispersion among the        detected signals before and after rotation transformation        (signal processing)

(A1) A difference (magnitude of movement) in dispersion between themeasured acceleration waveform (detected raw signal) in the Z-axisdirection (gravity component) or the Y-axis direction (dolly movingdirection) for each statistical processing storage period describedabove and the transformed standard measured acceleration (or dispersioncharacteristics among data in the statistical processing storage periodwhen a difference value between the measured acceleration waveform(detected raw signal) for each cycle and the transformed standardmeasured acceleration is regarded as one data) is statisticallyprocessed. It is then determined whether the statistically processeddispersion value falls within a predetermined threshold value.

(A2) A difference in random value (irregularities) between the measuredacceleration waveform (detected raw signal) in the Z-axis direction(gravity component) or the Y-axis direction (dolly moving direction) foreach statistical processing storage period and the transformed standardmeasured acceleration is used for behavior estimation. It is thendetermined whether the calculated random value falls within apredetermined threshold value.

The random value means the number of variations of the sign of the abovedifference value within the statistical processing storage period.Assume that the difference value between the measured accelerationwaveform (detected raw signal) in the k-th cycle and the transformedstandard measured acceleration is Δ(k). When “Δ(k)<Δ(k+1)”, D(k)=+1 isassigned. When “Δ(k)>Δ(k+1)”, D(k)=−1 is set. When “Δ(k)=Δ(k+1)”, D(k)=0is set. The total value of D(k) in the statistical processing storageperiod is defined as a random value. Consider the case where thesequence of {D(1), D(2), D(3), D(4), D(5), D(6), D(7), D(8), D(9)} is{+1, +1, +1, −1, −1, +1, +1, +1, +1}. In this sequence, the sign changesfrom +1 to −1 once and the sign changes from −1 to +1 once. Since thesign change is made two times in total, the random value is “2”.

If the frequency (similarity) that satisfies the above (A1) and (A2) atthe same time within a determination period of behavior estimation is80% or more, it is determined as matching. Assume that the cycle periodis 50 mS, the statistical processing storage period is 0.5 seconds, andthe determination period of behavior estimation is four seconds. Asdescribed above, in the present embodiment, the statistical processingstorage period is set sequentially while shifting every cycle period.The statistical processing storage period can thus be set only eightytimes (4000÷50) during the above determination period (four seconds) ofbehavior estimation. If, therefore, (A1) and (A2) are satisfied at thesame time with the frequency of 80% or more (64 times or more) in thestatistical processing results of eighty times, it is determined thatthe person being measured 2 is moving the dolly 20.

(B) Determination of similarities between the transformed standardmeasured acceleration and sample data

(B1) A difference (magnitude of movement) in dispersion between thetransformed standard measured acceleration in the Z-axis direction(gravity component) or the Y-axis direction (dolly moving direction) foreach statistical processing storage period and the standard measuredacceleration waveform (sample data) (or dispersion characteristics amongdata in the statistical processing storage period when a differencevalue between the transformed standard measured acceleration for eachcycle and the standard measured acceleration waveform (sample data) isregarded as one data) is statistically processed. It is then determinedwhether the statistically processed dispersion value falls within apredetermined threshold value.

(B2) A difference in random value (irregularities) between thetransformed standard measured acceleration in the Z-axis direction(gravity component) or the Y-axis direction (dolly moving direction) foreach statistical processing storage period and the standard measuredacceleration waveform (sample data) is used for behavior estimation. Itis then determined whether the calculated random value falls within apredetermined threshold value.

If the frequency (similarity) that satisfies the above (B1) and (B2) atthe same time within a determination period of behavior estimation is80% or more, it is determined as matching.

(C) Determination of expansion/contraction matching between thetransformed standard measured acceleration and sample data

An extension/contraction matching technique is applied between thetransformed standard measured acceleration in the Z-axis direction(gravity component) or the Y-axis direction (dolly moving direction) foreach statistical processing storage period and the standard measuredacceleration waveform (sample data). It is then determined whether theEuclidian distance (described in detail later) obtained for eachstatistical processing storage period falls within a predeterminedthreshold value. If the frequency (similarity) with which the Euclidiandistance falls within the threshold value in the determination period ofbehavior estimation is 80% or more, it is determined as matching.

The foregoing extension/contraction matching technique is a general termregarding a pattern matching technique to calculate a similarity betweenthe pattern of sample data (standard measured acceleration waveforms)and the pattern of detected signals (transformed standard measuredacceleration), taking into consideration partial extension/contractionon a time axis between these patterns.

In the system (behavior estimating system 52) according to the presentembodiment, the values (detected values) of detected raw signals WX(t),WY(t) and WZ(t) (at least one of these signals or a predeterminedfrequency component and a predetermined element component in a signal)vary with time. Since there is a difference in behavior speed (workingspeed) among persons being measured 2, the speed varying with timevaries from person being measured 2 to person being measured 2. Thus,the varying speed of detected raw signals for use in behavior estimationof human beings and animals and state estimation to estimate the stateof a predetermined object varies from person being measured to personbeing measured. Therefore, the use of the extension/contraction matchingtechnique in the estimation process in the system (behavior estimatingsystem 52) according to the present embodiment brings about theadvantage of absorbing the variations in varying speed of detected rawsignals (adapting to the extension/contraction of detected raw signalsin a direction in which time elapses) and thus improving estimationaccuracy.

The present embodiment will now be described by way of a specificexample of the positioning of the embodiment. First, a method forplanning an estimation process and business improvement 10 or providingservice using detected raw signals obtained while the person beingmeasured 2 shown in FIG. 1 is moving 22 the dolly, will be described.The present embodiment is not limited to these works but can be appliedto works such as manual operation 20, stopping 24 and walking 26 shownin (b) of FIG. 1 .

As a situation of use of the present embodiment shown in FIG. 1 , anexample of application to a warehouse management system (WMS) is shownin FIG. 11 . The WMS is roughly divided into mechanisms to solve twodifferent problems corresponding to “inventory control” and “workingsupport”. As an example of the inventory control, there are grasping ofbest before dates of stocks, grasping of stock position of each articlein a warehouse, etc.

As an example of an application situation of the working support isshown in FIG. 11 . An image marker MK is set for each of articles storedfor each of racks (shelves) RK arranged in the warehouse to indicate thecontents of each article. As an example of the image marker MK, thereare a one-dimensional or 2-dimensional direction code or a mark (icon,etc.) which allows the contents of an article to be easily distinguishedat first sight. The present embodiment is not limited to the code ormark. The image marker MK may be a communication node incorporating awireless communication function for each image marker MK and an imagemarker MK having placement information (position information) using abeacon.

A personal worker (PW) (or person being measured 2) chooses and extracts(picks) a designated one of the articles stored in each of the racks(shelves) RK and puts it in a cart (basket) CRT (or puts it on the dolly122 if it is a large article 8 to be carried).

Then, the personal worker (PW) (or person being measured 2) collects rawsignals detected from the activity meter 4 attached to the PW (1). Thecollected detected raw signals are stored temporarily in the memory unit(unit for storing waveform data output from the sensor). The detectedraw signals are read out to perform the foregoing steps (2) to (6). Theresults of the steps lead to business improvement (improvement inworking activity process and procedure) of each individual personalworker PW (or person being measured 2). The present embodiment is notlimited to these steps. The personal worker PW (or person being measured2) can be provided with service (7) such as an instruction about thenext activity using a glasses-type wearable terminal (drive device) 1100(refer to a detailed description using FIG. 38 et seq).

An example of use in WNS with reference to FIG. 1 has been described asan application example of the present embodiment. The present embodimentis not limited to the example but can be applied to a socialinfrastructure market, a personal market, a health care market, aproduction/management market (design/manufacturing/inspection site) andthe like.

As for the social infrastructure market, for example, the presentembodiment can be applied to construction environment such astransportation, bridges and buildings, a supply market of publicconsumer goods such as electricity, water and gas, a financial industrymarkets related to banking, securities, non-life insurance, etc. As forthe personal market, it can be applied to a medical practice and anursing care field.

The following is a description of each configuration of the behaviorestimating system 52. The hardware configuration of the signalprocessing system according to the present embodiment shown in FIG. 12includes the activity meter 4, a sensor device 6, an edge device 42 anda server 40. The sensor device 6 is fixed to a predetermined object(e.g., dolly 122) other than human beings, as will be described laterwith reference to FIG. 16 . Wired or wireless communication line isconfigured between the activity meter 4 and edge device 42, between thesensor device 6 and edge 42 device and between the edge device andserver 40 to allow information communications therebetween.

The edge device 42 and the server 40 cooperatively perform estimationresult processing and service providing control described later. Theedge device 42 also collects and organizes results (contents oftransmission information 30) of processing performed by the activitymeter 4 and a signal processing unit 60 in the interior of the sensordevice 6.

The functional configuration of the signal processing system shown inFIG. 12 includes the behavior estimating system 52, a service controlunit 58, a position detecting signal generation unit 46 and a controlunit 48. The activity estimating system 52 is configured by the activitymeter 4, the sensor device 6 and part of the edge device 42.

FIG. 12 mainly shows the configuration of the activity meter 4. Thehardware configuration 62 and functional configuration 64 of the sensordevice 6 basically coincide with the configuration of the activity meter4.

The activity meter 4 (and the sensor device 6) includes a fixing member50 and the signal detection unit 60. The fixing member 50 has a functionof fixing the signal detection unit 60 to part of the body of the personbeing measured 2 when the activity meter 4 is attached to the personbeing measured 2. Specifically, the fixing member 50 has a structure ofa wristband and can be attached directly to the arm or leg of the personbeing measured 2 as shown in (a) of FIG. 1 . The activity meter 4 is notlimited to this structure. For example, the activity meter 4 can bestuck on the human body using an adhesion layer and using part ofclothing to be attached to the human body, such as hats, glasses andshoes or part of clothes. As the fixing member 50 to fix the sensordevice 6 to the article 8 to be carried, an adhesive layer and part of astring that is used for packaging of the adhesive layer and the article8 to be carried.

In the embodiment shown in FIG. 12 particular, the whole signaldetection unit 60 is built into the activity meter 4 (or the sensordevice 6). The hardware configuration 62 of the signal detection unit 60includes a power generation unit (environmental vibration type orphotoelectric conversion type) 68, a power storage unit (battery) 66,the three-axis acceleration sensor 72, the three-axis geomagnetismsensor 74, a signal amplifier group 76, an A/D converter (analog todigital converter) 78, the control unit 80, the memory unit 82 and acommunication control unit 84.

As a type of the power generation unit 68, an environment electric typeas described later can be used and a photoelectric conversion type suchas a solar cell can be used. The power generated by the power generationunit 68 is stored in the power storage unit 66. Using the power storedin the power storage unit 66, the circuits of the signal detection unit60 are operated.

The three-axis acceleration sensor 72 can obtain acceleration signalwaveforms (WX(t), WY(t), WZ(t)) in three axis directions (X-axis, Y-axisand Z-axis directions) which are orthogonal to each other, and thethree-axis geomagnetism sensor 74 can detect a direction of thegeomagnetism. In one example of the present embodiment, the three axisdirections defined in the geomagnetism sensor 74 completely coincidewith the three-axis directions (X-axis, Y-axis and Z-axis directions) ofthe three-axis acceleration sensor 72. In other words, in the interiorof the signal detection unit 60 of the activity meter 4 and in thesignal detection unit 60 of the sensor device 6, the three axis (X axis,Y-axis and Z-axis) directions which are orthogonal to each other are setin advance.

For convenience of description, the activity meter 4 will be describedby way of an example of a three-axis sensor. However, the presentembodiment is not limited to the three-axis sensor, but for example, aone-axis sensor or a two-axis sensor can be used, or athree-or-more-axis sensor (e.g., a three-axis acceleration sensorfunction and a multi-axis angular velocity sensor function are includedin the same sensor) can be used.

The three-axis-direction detected raw signals (WX(t), WY(t), WZ(t))obtained from the three-axis acceleration sensor 72 and geomagnetismsensor 74 are each amplified by the signal amplifier group 76 and thenconverted to digital signals by the A/D converter (analog to digitalconverter) 78. The digital signals are stored temporarily in the memoryunit 82. The signal detection unit 60 includes the control unit 80 toperform signal processing using the detected raw signals (WX(t), WY(t),WZ(t)) stored in the memory unit 82. The results of the signalprocessing are transmitted to the edge device 42 as transmissioninformation 30 via the communication control unit 84.

The functional configuration 64 of the signal detection unit 60 will bedescribed below. In detected raw signal generation 90, a process ofextracting detected raw signals from the three-axis acceleration sensor72 and the three-axis geomagnetism sensor 74 and converting the detectedraw signals into digital signals by the A/D converter 78 through thesignal amplifier group 76, is performed.

The control unit 80 performs reference direction extraction 92 and angleextraction 94 based on results obtained in the detected raw signalgeneration 90 and detected signal generation (signal processing) 96after reduction of noise components, using the memory unit 82.

The reference direction extraction 92 means a function of extracting apredetermined reference (direction) using a result of the detected rawsignal generation 90. As an example of the function, a gravity direction(e.g., z-axis direction) can be extracted using a detected raw signalwaveform to be acquired from the three-axis acceleration sensor 72.Also, a geomagnetic direction (e.g., y-axis direction) can be detectedusing the raw signals detected from the three-axis geomagnetism sensor74.

The process of calculating angle θ between a predetermined reference(directions of z axis and y axis corresponding to the predeterminedreference) and the coordinate axis (X axis, Y axis and Z axis) preset inthe activity meter 4 and sensor device 6, corresponds to the angleextraction 94.

Furthermore, the detected signal generation (signal processing) 96 afterreduction of noise components, means a function of processing thedetected raw signals (WX(t), WY(t), WZ(t)) using a result of the angleextraction 94 to generate detected raw signals (Wx(t), Wy(t), Wz(t))after reduction of noise components. The reduced noise components of atleast one of the resulting detected raw signals (Wx(t), Wy(t), Wz(t))are less than those of the detected raw signals (WX(t), WY(t), WZ(t)).

Then, the result is transmitted from the activity meter 4 to the edgedevice 42 via the communication control unit 84. The transmissioninformation 30 to be transmitted at this time includes extracted angleinformation 32 and detected signal information 34 whose noise componentsare reduced.

The system according to the present embodiment shown in FIG. 12 includesthe communication control unit 84 together with the activity meter 4 andthe sensor device 6, each having a wireless communication function.Using the wireless communication function, the position of a personbeing measured can be detected in the same manner as a GlobalPositioning System (GPS).

The position detecting signal generation units 46 controlled by thecontrol unit for position detecting signal generation of the edge device42 are distributed in different locations of (A) to (D). Transmissiontime information is continuously transmitted wirelessly from thelocations of (A) to (D). A difference in time at which the communicationcontrol unit 84 receives time information from each of the locations of(A) to (D) is detected to make it possible to detect a place on whichthe activity meter 4 (or sensor device 6) is disposed in real time.

The combination of the position information and the detected signals(Wx(t), Wy(t), Wz(t)) whose noise components are reduced, obtained fromthe three-axis acceleration sensor 72 and the three-axis geomagnetismsensor 74, as described above, brings about the advantage that theposition can be detected with high accuracy.

The behavior estimation engine 56 of a behavior estimation unit 54 ofthe edge device 42 performs behavior estimation based on thetransmission information 30 transmitted from the sensor devices 4 and 6(or state estimation and request estimation of the person being measured2). Then, based on a result of the estimation, service providing andbusiness improvement 10 are proposed in the service control unit 58.

Specifically, when the result (behavior estimation result 16) obtainedfrom the behavior estimation engine 56 is transmitted to the server 40,the contents of a service instruction 11 is returned from the server 40.Based on the contents of the service instruction 11, a drive device 44is operated from the edge device 42 using wireless communication, andservice providing is performed to the person being measured 2 or otherusers.

The drive device 44 includes an electric motor or a light-emittingdevice, a display device, a speaker, etc., to drive the electric motor,perform emission and display, output voice, and the like in accordancewith an operation from the edge device 42.

FIG. 13 shows a system according to another embodiment regarding thesignal processing system shown in FIG. 12 . In the system (signalprocessing system) according to the embodiment shown in FIG. 13 , onlythe detected raw signal generation 90 and the position detection 98 areperformed in the activity meter 4 (or in the sensor device 6) in thefunctional configuration 64 of the position detection unit 60.

On the edge device 42 side, the foregoing reference direction extraction92, angle extraction 94 and detected signal extraction after noisecomponent reduction (signal Processing) 96, which are included in thefunctional configuration 64 of the signal detection unit 60, areperformed.

Adapting to the function sharing (function dispersion between theactivity meter 4/sensor device 6 and the edge device 42), only detectedposition information 38 and detected raw signal information 36 istransmitted from the activity meter 4 or the sensor device 6 to the edgedevice 42 as the transmission information 31.

As a description corresponding to the above function sharing, acommunication control unit 86 and a control unit 81 included in part ofthe hard configuration 62 of the signal detection unit 60, and thememory unit 83 are specified particularly in the edge device 42. Thoughnot shown explicitly in FIG. 12 , in the signal processing system shownin FIG. 12 , too, the edge device 42 includes the communication controlunit 86, control unit 81 and memory unit 83.

More specifically, in the system (signal processing system) according tothe embodiment shown in FIG. 13 , the detected position information 38and the detected signal information 36 transmitted from the activitymeter 4 (or the sensor device 6) are stored temporally in the memoryunit 83 through the communication control unit 86. Then, the controlunit 81 of the edge device 42 performs signal processing using thedetected raw signals (WX(t), WY(t), WZ(t)) stored in the memory unit 83.The detected signals (Wx(t), Wy(t), Wz(t)) whose noise components arereduced, obtained by the signal processing are also stored in the memoryunit 83.

Processing the detected raw signals (WX(t), WY(t) and WZ(t)) collectedfrom the three-axis acceleration sensor 12 and the three-axisgeomagnetism sensor 74 in the edge device 42, the functions of theactivity meter 4 and the sensor device 6 can be simplified. This bringsabout the advantage that not only the activity meter 4 or the sensordevice 6 alone, but also the entire signal processing system can be madeinexpensively.

Then, the behavior estimation engine 56 that is built into the behaviorestimation unit 54 of the edge device 42 performs a state estimationprocess or a behavior estimation process of the person being measured 2and a request estimation process (or the person being measured 2 or theother users) using the detected signals (Wx(t), Wy(t), Wz(t)) whosenoise components are reduced, which are stored in the memory unit 83.

The contents of the hardware configuration 62 (communication controlunit 86, control unit 81 and memory unit 83) belonging to the signaldetection unit 60 set in the edge device 42 also serve as the hardwareconfiguration 63 that performs the process of the behavior estimationunit 54.

Then, the behavior estimation result 16 obtained from the behaviorestimation engine 56 of the behavior estimation unit 54 of the edgedevice is transmitted to the server 40. Then, the service 40 uses theresult to examine the contents of service providing. After that, theservice 40 performs service instruction 11 to the edge device 42.

The contents other than those described above coincide with the contentsthat have already been described with reference to FIG. 12 .

As described with reference to FIG. 12 , the three-axis (X axis, Y axisand Z axis) directions orthogonal to each other are preset in theinterior of the signal detection unit 60 of the activity meter 4 and inthe interior of the signal detection unit 60 of the sensor device 6. Therelationship between the three-axis directions and predeterminedcriteria (e.g., the gravity direction along the z axis and thegeomagnetism direction along the y axis) will be described in detailwith reference to FIGS. 14, 15 and 16 .

For example, as shown in FIG. 14 , the direction toward the middlefinger of the person being measured 2 from the arm 120 thereof equippedwith the activity meter 4 is set to a Y axis. For convenience ofdescription, the direction of action 100 applied when the person beingmeasured 2 pushes the dolly (the direction in which the person beingmeasured 2 exerts force) is caused to coincide with the Y-axisdirection.

The axial direction perpendicular to the back of a hand of the personbeing measured is set as the Z-axis direction and the directionorthogonal to the Y and Z axes (direction from the little finger of theperson being measured 2 toward the thumb thereof) is set as the X axis.In accordance with these coordinate axes, the three-axis accelerationsensor 72 outputs acceleration waveforms (detected raw signals WX(t),WY(t) and WZ(t)) in the X-axis direction, Y-axis direction and Z-axisdirection.

At the same time, the three-axis geomagnetism sensor 74 outputs fieldintensity signals in the X-axis, Y-axis and Z-axis directions (which canbe described as WX(t), WY(t) and WZ(t) that mean the detected rawsignals or (WX*(t), WY*(t), WZ*(t)) to be distinguished from theacceleration waveforms).

On the other hand, the direction along criterion A (geomagnetismdirection) 102 is defined as the y axis (direction from the South Pole114 to the North Pole 112). The direction from the west to the east isalso defined as the x axis. In most cases, the Y and y axes do notcoincide with each other, and the waveforms of Wx(t) and Wy(t) are notdirectly detected as raw signals because the Y and y axes are inclinedto each other.

For convenience of description, a case where the X-axis and x-axiscoincide with each other will be described with reference to FIG. 15 .When the dolly 122 is moved toward the north on the floor surface 128,the movement direction of the dolly 122 coincides with the y-axisdirection. If the z axis is defined with the gravity direction ascriterion B_104, the y-axis and z-axis are orthogonal to each other.

As shown in FIG. 15 , the direction in which the person being measured 2pushes the dolly 122 (action direction 100 of the Y axis) has aninclination of θyz relative to a surface 118 perpendicular to thegravity. Since the Y-axis direction and Z-axis direction are orthogonalto each other as described above, an inclination angle between the zaxis and Z axis is also θyz.

The time change of a virtual acceleration waveform Wy(t) to be obtainedin the y-axis direction is shown on the right side of FIG. 15 . When theperson being measured 2 first start to push the dolly 122 in thestationary state (Wy(t)=0), the Wy(t) waveform rises as time elapses inaccordance with the start of movement of the dolly 122.

The time change of a virtual acceleration waveform to be obtained in thegravity direction (z-axis direction) is represented by Wz(t). Sinceconstant gravitational acceleration is always applied to the activitymeter 4, a DC component corresponding to gravitational acceleration G isalways added to Wz (t).

If there is an irregularity in the floor surface 128 on which the dolly122 moves, an acceleration component (noise component) corresponding tothe irregularity is mixed in Wz(t) in response to the movement of thedolly 122.

The Wy(t) waveform or Wy(t) waveform shown on the right side of FIG. 15cannot be collected directly as a detected raw signal from thethree-axis acceleration sensor 72 (or three-axis geomagnetism sensor74). If, however, the above angle θyz is used, it is possible tocalculate Wz(t) and Wy(t) from the detected raw signals WZ(t) and WY(t)that can be collected directly from the three-axis acceleration sensor72 (or three-axis geomagnetism sensor 74).

In the present embodiment (signal processing system), criterion A(geomagnetism direction) 102 and criterion B (gravity direction) 104 areextracted using raw signals WX(t), WY(t) and WZ(t) detected fromdifferent sensors of the sensor device (activity meter) 4 and the sensordevice 6. To do this, it is necessary to extract a predeterminedcriterion (direction) with high accuracy (by reducing noise componentsmixed in the detected raw signals).

An example of application to FIG. 15 is shown in FIG. 16 . In FIG. 15 ,signal processing/behavior estimation of the person being measured 2 isperformed using the raw signals detected from the activity meter 4attached to the arm 120 of the person being measured 2 to lead tobusiness improvement (improvement in working process) through thebehavior history analysis of the person being measured 2 and to providenecessary services. The behavior estimation of the person being measured2 is not limited to this. For example, the sensor device 6 can be fixedto part of a predetermined object such as the dolly 122 (by pasting orthe like), and signal processing/behavior estimation is performed by thedetected raw signals alone collected from the sensor device 6 to performa behavior history analysis/business improvement and provide services.In addition, the raw signals detected from the activity meter 4 and theraw signals detected from the sensor device 6 can be combined. Thiscombination brings about the advantage of increasing the behaviorestimation of the person being measured 2 to allow more appropriatebusiness improvement planning and service providing.

It has been described with reference to FIGS. 12 and 13 that in thesystem of the present embodiment (signal processing system), the edgedevice 42 operates the drive device 44 to allow service to be provided(to the person being measure 2 or other users). To provide the service,an electric motor (drive device) 124 corresponding to the drive device44 and a light emitting device or a loudspeaker (drive device) 126 areprovided.

The light emitting device emits light or the loudspeaker outputs voiceas the electric motor 124 moves based on the service instruction 11 fromthe server 40, which has determined “the person being measured 2 ismoving the dolly 20” as the behavior estimation result 16 in thebehavior estimation engine 56 (FIGS. 12 and 13 ). When the electricmotor 124 automatically starts to move during the dolly movement 20 ofthe person being measured 2, the advantage capable or reducing theburden on the person being measured 2 who is moving the dolly isproduced. The light emission of the light emitting device or the voiceoutput from the loudspeaker makes it possible to automatically “callattention” to a third party as the dolly moves. This also brings aboutthe advantage that the dolly can be moved safely (prevention ofinadvertent contact with a third party).

A description of a route through which noise components are mixed intothe detected raw signals will be given first. As shown in FIG. 12 or 13, in the interior of the signal detection unit 60, faint signalsobtained from the sensors 72 and 74 are amplified by the signalamplification group 76 and converted into digital signals through theA/D converter 78.

Below is a description of a method of generating signal components in apredetermined direction from raw signals WZ(t), WY(t) and WX(t) detectedfrom the sensors 72 and 74 using angle θ extracted by the above method.

FIG. 17 shows a change over time in signals detected directly orindirectly when the person being measured 2 pushes the dolly 122 and thedolly 122 starts to move. For the sake of convenience, assume here a wayof taking the same coordinate axis as in FIG. 16 (a case where the X andx axes coincide with each other).

When the dolly 122 starts to move, the acceleration waveform Wy(t) inthe moving direction (y-axis direction) of the dolly increases as timeelapses as shown in (a) of FIG. 17 . On the other hand, the accelerationwaveform Wz(t) in the gravity direction (z-axis direction) is added to agravity acceleration component G (DC component) that always exists, andan acceleration change corresponding to the irregularities of the floorsurface 128 appears ((b) of FIG. 17 ).

In addition, as described above with reference to FIG. 15 , the Y-axisdirection set in advance in the activity meter 4 inclines only angle θyzrelative to the surface 118 perpendicular to the gravity. Thus, thenoise component of Wz(t) (corresponding to the irregularities of thefloor surface 128) is mixed into the detected raw signal WY(t) inaccordance with the angle θyz.

When the person being measured 2 of average height pushes the dolly 122,θyz is represented by θs. The detected raw signal “WY(t)|θyz=θs”collected from the Y-axis direction of the acceleration sensor 72 of theactivity meter 4 when θyz is equal to θs have a waveform shown in (d) ofFIG. 17 .

When the person being measured 142 who is smaller than average pushesthe dolly 122, θyz is smaller than θs. In this case, the amount ofmixing of the noise component of Wz(t) (corresponding to theirregularities of the floor surface 128) into a detected raw signal“WY(t)|θyz<θs” is relatively small, with the result that the waveformshown in (c) of FIG. 17 is obtained.

When the person being measured 144 who is taller than average pushes thedolly 122, θyz is larger than θs. In this case, the amount of mixing ofthe noise component of Wz(t) (corresponding to the irregularities of thefloor surface 128) into a detected raw signal “WY(t)|θyz>θs” isrelatively large, with the result that the waveform shown in (e) of FIG.17 is obtained.

In the present embodiment, some signal processing is applied to thedetected raw signal shown in (c) or (e) of FIG. 17 to reduce theinfluence of the height of the person being measured 2. Performing thebehavior estimation (or situation estimation) of the person beingmeasured 2 using a result of the signal processing to reduce theinfluence of the height of the person being measured 2 as describedabove, brings about the advantage of increasing the determinationaccuracy of the behavior estimation (or situation estimation).

As a method to reduce the influence of the height of the person beingmeasured 2, the detected raw signal shown in (c) or (e) of FIG. 17 canbe converted into a detected signal “WY(t)|θyz=θs” when θyz is equal toθs to perform behavior estimation (or situation estimation).

The present embodiment is not limited to the foregoing method. Forexample, the detected raw signal shown in (c) or (e) of FIG. 17 can beconverted into a detected raw signal Wy(t) when θyz is equal to 0 toperform behavior estimation (or situation estimation).

In the system (behavior estimating system 52) according to the presentembodiment, behavior estimation is performed by matching a waveform(correct waveform) to be a basis of the behavior estimation and theactual waveform, but specifically, any extension/contraction matchingtechnique can be adopted. As an example of the extension/contractionmatching technique, a DP matching technique (dynamic programmingmatching technique) will be described with reference to FIGS. 18 to 20 .Note that the system (signal processing system) of the presentembodiment is not limited to the matching technique, but any matchingtechnique can be used for the estimation process.

An example of the standard measured acceleration waveform (sample data)which is measured in advance is shown in (a) of FIG. 18 . When theperson being measured 2 pushes the dolly 122, which is in a stationarystate at the beginning, and the dolly 122 starts to move, theacceleration in the moving direction (y-axis direction) of the dolly 122(y-axis direction) increases from “0” as time elapses. When theacceleration becomes closer to a fixed one, the acceleration to bedetected decreases ((a) of FIG. 18 ).

The horizontal axis memory of FIG. 18 represents a cycle period (e.g.,50 mS). An expedient value of the standard measured accelerationwaveform (sample data) collected for each cycle period is explicitlyshown in each graph of FIG. 18 .

Consider the case where the person being measured 2 of the heightdifferent from the average height pushes the dolly 122 slowly. Thebehavior speed (work speed) at which the person being measured 140 ofaverage height starts to push the dolly 122 to collect the standardmeasured acceleration waveform (sample data) in advance, is defined as“standard speed”. The detected raw signal WY(t) in the Y-axis directionwhen the behavior speed (work speed) at which the person being measured2 starts to push the dolly 122 is slower than the “standard speed” isshown in (b) of FIG. 18 .

As described with reference to FIG. 2 , the low-frequency component (DCcomponent) of Q42L due to gravity is always superimposed on the detectedraw signal WY(t) in the Y-axis direction. As compared with the standardmeasured acceleration waveform (sample data) ((a) of FIG. 18 ), theperson being measured 2 pushes the dolly 122 slowly this time.Therefore, the amount of change in acceleration in (b) of FIG. 18 issmaller than the standard measured acceleration waveform (sample data).

After the gravity component Q42L is subtracted from the detected rawsignal WY(t) in (b) of FIG. 18 , it is converted into the convertedstandard measured acceleration using the equation (5) or (7). Theresultant amplitude value is standardized so as to coincide with that in(a) of FIG. 18 , and the characteristics of the standardized amplitudevalue are shown in (c) of FIG. 18 .

The maximum signal amplitude in (c) of FIG. 18 after the amplitude valueis standardized coincides with the standard measured accelerationwaveform (sample data) in (a) of FIG. 18 . Since, however, the personbeing measured 2 pushes the dolly 122 slowly, the change in signal inthe time axis direction is small (slow).

In the extension/contraction matching technique employed in the system(behavior estimating system 52) according to the present embodiment, adifference in extension/contraction in time axis direction between (a)and (c) of FIG. 18 can automatically be corrected to calculate a degreeof matching between them. Since, therefore, pattern matching can beperformed while correcting a difference in behavior speed of the personbeing measured 2 or a difference in processing speed corresponding to afixed object obtained by the sensor device 6, the advantage that theaccurate pattern matching can be determined irrespective of thevariations of the behavior speed (processing speed) is brought about.

The waveform of (c) of FIG. 18 is transferred in the horizontal-axisdirection of FIG. 19 and the waveform of (a) of FIG. 18 is transferredin the vertical-axis direction of FIG. 19 . In FIG. 19 , the patternamplitude in the horizontal-axis direction and the pattern amplitude inthe vertical-axis direction coincide with each other. In the DP matchingtechnique (described above) which is a method of comparing similaritiesof data in series, a connection (arrow) between intersections at whichthe amplitude values are similar to each other is considered to be theoptimal route, and a sequential search is made for the optimal route.This search for the optimal route corresponds to a process of absorbingvariations in the behavior speed (processing speed).

A method for searching for the foregoing optimal route will be describedbelow. FIG. 20 is an enlarged view of the “enlarged area” shown in FIG.19 . In the example of FIG. 20 , when the elapsed time advances only acycle period, the amplitude changes from “7” to “11” in thehorizontal-axis direction, whereas the amplitude changes from “8” to“13” in the vertical-axis direction. At this time, there are threecandidates for the route. Specifically, the route (1) is a movement inthe horizontal direction, the route (2) is a movement in the obliquelyupward direction, and the route (3) is a movement in the verticaldirection.

First, a value of the square of the amount of deviation from the sampledata that has passed through the route (1) is calculated. The amplitudevalue in the vertical-axis direction after it has passed through theroute (1) remains “8”. On the other hand, the amplitude value in thehorizontal-axis direction changes from “7” to “11”. As a result, thesquare of the amount of deviation of the amplitude value after it haspassed through the route (1) is 9 (=(8−11)²).

Then, in the movement in the diagonally upward direction in the route(2), the amplitude value in the horizontal-axis direction after it haspassed through the route (2) becomes “11” and the amplitude value in thevertical-axis direction becomes “13”. As a result, the square of theamount of deviation between the two amplitude values is 4 (=(13−11)²).

Similarly, when it has passed through the route (3), the amplitude valuein the horizontal-axis direction remains “7”, but the amplitude value inthe vertical-axis direction changes from “8” to “13”. As a result, thesquare of the amount of deviation between data after it has passed is 36(=(13−7)²).

In this DP matching technique, a route in which the square value of theamount of deviation after it has passed through the route is thesmallest, is selected automatically. In the example shown in FIG. 20 ,therefore, the route (2) with the smallest square value “4” of theamount of deviation is automatically selected. As the optimal route isautomatically selected in this way, matching between patterns in whichthe extension/contraction varies in the time axis direction can be takenas in the relationship between (a) and (c) of FIG. 18 .

The following is a description of a method for calculating an amount oferror between the standard measured acceleration and the standardmeasured acceleration waveform (sample data), which have beentransformed during the statistical processing storage period. The squarevalues of the amount of deviation during the optimal route passing,which are calculated for each square (cycle period) shown in FIG. 20 ,are totaled in the statistical processing storage period. The value ofthe root of the total value is called “Euclidean distance” indicatingthe degree of pattern matching. In the example of FIG. 19 , theEuclidean distance is 2.83.

When there are two or more behavior estimation candidates for the personbeing measured 2 (standard measured acceleration waveforms (sample data)of different types are prepared), the Euclidean distance between thetransformed standard measured accelerations for each of the differentstandard measured acceleration waveforms (sample data). It is determinedthat the behavior of the person being measured 2 is similar to thebehavior estimation candidate with the shortest Euclidean distance.

On the other hand, in the case where behavior estimation as to whetherthe behavior of the person being measured 2 coincides with the dollymovement 22 is performed as shown in (b) of FIG. 1 , only the standardmeasured acceleration waveform (sample data) is prepared in advance.Therefore, in this case, the Euclidean distance between the abovestandard measured acceleration waveform (sample data) and thetransformed standard measured acceleration is calculated to performbehavior estimation (or an estimation process such as state estimation)according to whether the value of the calculated Euclidean distance is apredetermined threshold value or less.

Another embodiment in which the behavior estimating system 52 describedwith reference to FIGS. 12 and 13 is achieved by another hardwareconfiguration, is shown in FIG. 21 . The behavior estimating system 52includes a signal detection unit 60 and a behavior estimation unit 54.The signal detection unit 60 includes a memory unit (output waveformdata storage unit) 82 and some of an output waveform data/accelerationextraction unit 160, an angle extraction unit 164, a coordinatetransformation unit 166, a filter unit 162 and a control unit (controlmanagement unit) 80.

Also, the behavior estimation unit 54 includes a standardperson-being-measured data (sample data targeted for matchingcomparison) storage unit 170 and some of a feedback data storage unit176, a feedback data processing unit 178, a threshold value changedetermination unit 182, a threshold value change unit 184, an operationdetermination unit 172, an operation determination threshold valuestorage unit 174 and a control unit (control management unit) 80.

The memory unit (output waveform data storage unit) 82 stores data(detected raw signals) measured by the activity meter 4. The memory unit(output waveform data storage unit) 82 corresponds to the memory unit 82shown in FIG. 12 or 13 .

The output waveform data/acceleration extraction unit 160 fulfills thefunction of the detected raw signal generation 90 shown in FIG. 12 or13. A specific hardware configuration regarding the output waveformdata/acceleration extraction unit 160 corresponds to the three-axisacceleration sensor 72, signal amplification group 76 and A/D converter78 shown in FIG. 12 or 13 .

On the other hand, the function of the filter unit 162 coincides withthat of the low-pass filtering process 138 described with reference toFIG. 12 . In other words, the filter unit 162 extracts gravitationalacceleration components Q41L, Q42L and Q43L from the acceleration valuesoutput from the output waveform data/acceleration extraction unit 160.As the contents of the low-pass filtering process, prior art low-passfiltering characteristics can be used. The present embodiment is notlimited to the low-pass filtering characteristics. For example, afiltering method to acquire the average value can be used. The functionof a combination of the angle extraction unit 164 and the filter unit162 shown in FIG. 21 corresponds to the reference direction extraction92 and the angle extraction 94 shown in FIG. 12 or 13 . In other words,the angle extraction unit 164 calculates angle θyz of the arm 120 topush the dolly 122 of the person being measured 2 (see FIG. 16 ). Forthis angle extraction, as described above, the angle θyz is calculatedusing a trigonometric function and the gravitational componentscontained in the acceleration values measured by the wristband-shapedactivity meter (sensor device) 4 attached to the arm 120 of the personbeing measured.

The coordinate transformation unit 166 also fulfills the function of thedetected signal generation (signal processing) 96 from which noisecomponents are reduced in FIG. 12 or 13 . In other words, the measuredacceleration waveforms (detected raw signals) WX(t), WY(t) and WZ(t)obtained from the three-axis acceleration sensor 72 are rotationallytransformed into standard measured acceleration (signal processing)using equation (5) or (7). In this case, angle dθyz calculated from theangle extraction unit 164 using a gravity value is utilized.

The storage unit 170 of the standard person-being-measured data (sampledata targeted for matching comparison) stores the standard measuredacceleration waveforms (sample data).

The feedback data storage unit 176 stores data output from the feedbackdata processing unit, which will be described later.

The error characteristics (or the Euclidean distance characteristics forthe standard measured acceleration waveforms (sample data)) between thetransformed standard measured acceleration and measured accelerationwaveform (detected raw signal) with respect to angle θ(θyz) of the arm120 of the person being measured 2 have been described with reference toFIG. 8 . The characteristics are stored in the feedback data storageunit 176.

The operation determination unit 172 determines whether the person beingmeasured 2 performed the dolly movement 20 at a measurement target time.The characteristics shown in FIG. 8 are used for the determination, andinformation of the determination threshold value 150 during the behaviorestimation of the person being measured is used for the criterion of thedetermination.

The information of the determination threshold value 150 during thebehavior estimation of the person being measured is stored in theoperation determination threshold value storage unit 174.

As described with reference to FIG. 8 , the Euclidean distance that iscompared with an amount of error between the transformed standardmeasured acceleration and measured acceleration waveform (detected rawsignal) with respect to angle θyz of the arm 120 of the person beingmeasured 2 or the standard measured acceleration waveform (sample data)is calculated as appropriate in real time for each cycle. The thresholdvalue change determination unit 182 monitors in real time (for eachcycle) whether the above amount of error or Euclidean distance exceedsthe determination threshold value 150 during the behavior estimation ofthe person being measured. If the above amount of error or Euclideandistance exceeds the determination threshold value 150, information istransmitted to the threshold value change determination unit 182 tochange the determination threshold value 150.

Then, the threshold value change unit 184 controls a change of thedetermination threshold value 150 during the behavior estimation of theperson being measured, based on the information transmitted from thethreshold value change determination unit 182.

As an example of signal processing to be performed in the system (signalprocessing system) according to the present embodiment, a method forextracting angle θyz of the arm 120 of the person being measured 2 topush the dolly 122 using the detected raw signals WX(t), WY(t) and WZ(t)collected from the three-axis acceleration sensor 72 built in thewristband-shaped activity meter (sensor device) 4 and rotationallytransforming it into a detected signal to be obtained when θyz becomesequal to θs, has been described.

At the beginning of this specification of the present embodiment, thebasic contents of the present embodiment were itemized as (1) to (7).The following is a description of an example of a method forspecifically implementing the contents by the behavior estimating system52 shown in FIG. 21 .

As described above as a method for using results of behavior estimationof the person being measured 2, (7) service can be provided and (6)business (working process) improvement can be proposed. To (7) provideservice, real-time property is required for the above behaviorestimation.

On the other hand, when (6) business (working process) improvement isproposed, no real-time property is required but the following batchprocess can be performed. First, a method for performing the behaviorestimation with the batch process will be described. As an example ofthe behavior of the person being measured 2, a description of pushingthe dolly 122 by a person being measured with the activity meter 4attached to the arm 120, will be given.

Then, the processes performed up to the behavior estimation of theperson being measured 2 or a series of processes leading to (6) thebusiness improvement (improvement of working process) can be carried outcollectively (batch process) by reading data from the memory unit(output waveform data storage unit) 82 (at a later date) after the datefrom the collection date of behavior data of the person being measured2.

During the above batch process, the control unit (control managementunit) 80 reads the detected raw signals WX(t), WY(t) and WZ(t) from thememory unit (output waveform data storage unit) 82 and transfers them tothe angle extraction unit 164.

The angle extraction unit 164 has a function of the low-pass filteringprocess 138 described with reference to FIG. 3 to calculate values ofQ41L, Q42L and Q43L. Then, it calculates θyz and the like using theequations (1) to (3) to obtain dθyz and the like from FIG. 2 .

The coordinate transformation unit 166 rotationally transforms thedetected raw signals WX(t), WY(t) and WZ(t) (signal processing) usingthe equation (5) derived from FIG. 4 or the equation (7) derived fromFIGS. 4, 9 and 10 to calculate the transformed standard measuredaccelerations WXs(t), WYs(t) and WZs(t).

As a method for estimating the behavior of the person being measured 2,the methods (A) to (C) (or a combination of the methods) have alreadybeen described. Here, an example of using, for the behavior estimation,determination of dispersion and irregularities of the detected signalsbefore and after the (A) rotation transformation (signal processing),will be described.

The operation determination unit 172 shown in FIG. 21 compares a presetpredetermined threshold value with an error between the measuredacceleration waveforms (detected raw signals) WX(t), WY(t) and WZ(t) andthe standard measured accelerations WXs(t), WYs(t) and WZs(t) obtainedafter the rotation coordinate transformation. When the amount of errorfalls within the predetermined threshold value, the operationdetermination unit 172 determines that the person being measured 2 ismoving the dolly 122. When it exceeds the predetermined threshold value,the unit 172 determines that the person being measured 2 is not movingthe dolly 122.

Simultaneously with the above, the feedback data processing unit 178extracts the relationship in the amount of error between the measuredacceleration waveforms (detected raw signals) WX(t), WY(t) and WZ(t) andthe standard measured accelerations WXs(t), WYs(t) and WZs(t) obtainedafter the rotation coordinate transformation, for each angle θ(θyz) thatis formed between the surface 118 perpendicular to the gravity and theaction direction (Y-axis direction) 100. A result of the extraction isstored in the feedback data storage unit 176.

Since the foregoing process is a batch process, continuous data of theperson being measured 2 for two or three hours, which was measured theday before, or data for one day can collectively be recorded at a timein the feedback data storage unit 176.

After that (at a later data after the date when the extraction result isrecorded in the feedback data storage unit 176), the feedback dataprocessing unit 178 statistically analyzes the data recorded in thefeedback data storage unit 176 to create a characteristic graph shown inFIG. 8 .

In FIG. 8 , the determination threshold value 150 at the time ofbehavior estimation of the person being measured is a constant valueirrespective of angle θ. However, the determination threshold value 150is not limited to the constant value. The determination threshold value150 can be reset as appropriate according to angle θ.

If the determination threshold value 150 is reset in accordance withangle θ, the threshold value change unit 184 sets the changed thresholdvalue. The changed threshold value corresponding to the reset angle θ isstored in the operation determination threshold value storage unit 174as appropriate. The threshold value change determination unit 182determines whether the threshold value change unit 184 needs to resetthe threshold value. When the threshold value needs to be reset, itsinformation is transmitted from the threshold value change determinationunit 182 to the threshold value change unit 184.

Another embodiment regarding a flow of the process to be performed bythe behavior estimating system 52 shown in FIG. 21 will be describedwith reference to FIGS. 22A to 22C.

When the behavior estimating system 52 starts a process in Step S, theperson being measured 140 of average height measures accelerationwaveform data obtained when he or she moved the dolly 122 and stores itin the storage unit 170 of the standard person-being-measured data(sample data targeted for matching comparison) as a standard measuredacceleration waveform (sample data) in Step A1.

After that, in Step A2, the output waveform data/acceleration extractionunit 160 acquires a detected raw signal (measured acceleration waveform(detected raw signal)) from the person being measured 2.

In Step B1, the filter unit 162 extracts the decomposed components (G41Land G42L) of the gravitational acceleration (see FIG. 3 ). In Step B2,the angle extraction unit 164 calculates inclination angle θ(θyz) of thearm 120 of the person being measured 2 from Q41L and Q42L obtainedabove. For this calculation, the equation (1), (2) or (3) derived fromFIG. 2 is used. As a method for calculating angle θ(θyz), one of“setting 1: a method for calculating θ(θyz) using the equation (1)”,“setting 2: a method for calculating θ(θyz) using the equation (2)” and“setting 3: a method for calculating θ(θyz) using the equation (3)” canbe used. The angle calculating method is not limited to the above, butthe following can be used: “setting 4: a method using the average valuebetween θ(θyz) values obtained from the equations (1) and (2)” and“setting 5: a method using the average value among θ(θyz) valuesobtained from the equations (1), (2) and (3).

In Step B3, the coordinate transformation unit 166 rotationallytransforms the measured acceleration waveforms (detected raw signals)WX(t), WY(t) and WZ(t) into the transformed standard measuredaccelerations (signal processing). For the rotation transformation(signal processing), the foregoing equation (5) or (7) is used.

As has already been described, in the system (behavior estimating system52) according to the present embodiment, there is a relationship of“determination period of behavior estimation” 2 “statistical processingstorage Period” Z “cycle period”. One process flow from Step A2 to StepB3 in FIG. 22A means a process of one cycle. Thus, in order to estimatethe behavior of the person being measured 2 in the behavior estimatingsystem 52, it is necessary to repeat the cycle at least the number oftimes corresponding to the “determination period of behavior estimation”at the beginning (for example, when “determination period of behaviorestimation” is four seconds and the “cycle period” is 50 mS, it isnecessary to repeat the process flow from Step A2 to Step B3 at leasteighty times at the beginning).

In Step B5, in response to the above, it is determined whether the abovecycle has been repeated only a predetermined number of times. If thenumber of times does not reach the predetermined number of times, theprocess flow is advanced (incremented) by one cycle in the outputwaveform data/acceleration extraction unit 160 to start the next processflow (cycle) from Step A2 to Step B3 (Step B10). On the other hand, ifthe behavior estimation process can be performed after the cycle isrepeated a predetermined number of times, the process flow proceeds toStep B4.

As an example of the behavior estimation process of the person beingmeasured 2 to be performed in Step B4 (in the case of (A) below), theoperation determination unit 172 determines whether an error between themeasured acceleration waveform (detected raw signal) and the transformedstandard measured acceleration satisfies conditions for the thresholdvalue. However, as a specific behavior estimation processing methodperformed in the system (behavior estimating system 52) according to thepresent embodiment, one of the following methods (described above) or acombination thereof can be adopted:

-   -   (A) Determination of dispersion and irregularity between the        detected signals before and after the rotation transformation        (signal processing);    -   (B) Determination of similarity between the transformed standard        measured acceleration and the sample data; and    -   (C) Determination of extraction/contraction matching estimation        of scaling between the transformed standard measured        acceleration and the sample data.

If the error does not satisfy the condition for the threshold value as aresult of the above determination of the behavior estimation, theoperation determination unit 172 determines that the person beingmeasured 2 is not moving the dolly 122 at the target time (Step B7).

On the other hand, when the value of the error satisfies the conditionfor the threshold value as a result of the determination in Step B6, theoperation determination unit 172 determines that the dolly is moving atthe target time (Step B8). Though not shown, when the behavior of theperson being measured 2 is estimated as the dolly movement 20, the edgedevice 42 and the server 40 propose the business improvement 10 incooperation with each other if necessary, based on the service providingand the behavior history of the person being measured 2 (FIG. 1 andFIGS. 11 to 13 ).

The process from Step A2 to Step B7/B8 is repeated until the behaviorestimation process or the service providing process is completed. Inother words, it is determined in Step B9 whether the process iscompleted in a section corresponding to the measured data, and when thebehavior estimation process or the service providing process iscompleted, a series of processes is terminated (Step E). When thebehavior estimation process or the service providing process is notcompleted, the flow returns to Step A2 via Step B10.

Furthermore, in association with the series of processes, thedetermination threshold value 150 is changed as appropriate during thebehavior estimation of the person being measured as described withreference to FIG. 8 (FIG. 22C).

Performing the above series of processes in FIG. 22B, the feedback dataprocessing unit 178 calculates the characteristics of an amount of error(Euclidean distance calculated as compared with the standard measuredacceleration waveform (sample data)) between the measured accelerationwaveform (detected raw signal) and the transformed standard measuredacceleration for the angle θ(θyz) of the arm 120 of the person beingmeasured 2 shown in FIG. 8 . The result is then recorded in the feedbackdata storage unit 176 (Step C1).

Then, the feedback data processing unit 178 reads the abovecharacteristic data from the feedback data storage unit 176 to perform astatistical process for each angle θ(θyz) and calculate a correctedvalue of the determination value 150 (matching threshold value) duringthe behavior estimation of the person being measured from the result.The calculated corrected value is stored in the feedback data storageunit 176 (Step C2).

The threshold value change determination unit 182 reads the result outof the feedback data storage unit 176 and determines whether the angleerror for each angle θ(θyz) exceeds a set value (for example, 15%) (StepC3). If it does not exceed the set value, any special process is notperformed but the process after Step B4 is continued.

If it exceeds the set value (determination result in Step C3), thethreshold value change unit 184 changes the correction of the thresholdvalue (determination threshold value 150 during the behavior estimationof the person being measured) to be used for matching determinationaccording to the extracted angle θ(θyz) (Step C4).

The foregoing descriptions given with reference to FIGS. 22A to 22C arebased upon an example of behavior estimation at the time of the dollymovement 20 of the person being measured 2. However, the system(behavior estimating system 52) according to the present embodiment isnot limited to the dolly 20 of the person being measured 2, but can beapplied to the behavior estimation for any other behavior. Furthermore,the system of the present embodiment is not limited to the specificperson being measured 2. For example, as shown in FIG. 16 , stateestimation (estimation of the state of a predetermined object) can beperformed using a signal detected from the sensor device 6 fixed to apredetermined object ((a) in FIG. 25 ) such as a portion of the dolly122 and the article 8 to be carried.

In order to form the angle θyz in FIGS. 2 and 15 , the coordinate axes(Y and Z axes) preset in the interior of the wristband-shaped activitymeter (sensor device) 4 are defined as a reference line. As the rotationangle from the reference line (Y and Z axes), the angle θyz is set inthe counterclockwise direction. However, the angle θyz is not limited tothe direction. Angle θyz′ can be set in the clockwise direction as shownin FIGS. 23 and 24 . Similarly, the angle θyz can be set in thecounterclockwise direction with the z and y axes along the gravitydirection and the floor direction as a reference line.

An application example other than the embodiment focused on the dollymovement 20 described so far, will be described below. An example astate estimation method in which the person being measured 2 carries thearticle 8 directly by hands without using the dolly 122, is shown in (a)of FIG. 25 .

In FIG. 25 , too, a coordinate axis direction similar to those in FIGS.14 to 16 is defined. In other words, in the wristband-shaped activitymeter (sensor device) 4 attached to the person being measured 2, thedirection from the arm 120 of the person being measured to the fingersthereof is defined as a Y axis, the direction orthogonal to the Y axisand from the left little finger to the thumb is defined as an X axis,and the axis direction perpendicular to the back of the hand is definedas a Z axis.

Furthermore, the direction of criterion B (gravity) 104 is defined as az axis and the direction of criterion A (geomagnetism direction) 102 isdefined as a y axis. The criterion B (gravity) 104 and criterion A(geomagnetism direction) 102 are extracted from the detected raw signalscollected from the three-axis acceleration sensor 72 and the three-axisgeomagnetism sensor 74 which are built in the wristband-shaped activitymeter (sensor device) 4 through the same method as described above. Forbrevity of description here, the x-axis direction and the X-axis areprovisionally caused to coincide with each other.

When the person being measured 2 walks empty-handed without carrying anyspecific article, the Y-axis direction is closer to the z-axisdirection. As compared with this, when the person being measured 2 walkswith a specific article 8 to be carried, the Y axis is directed to thevertical direction of the z axis. If, therefore, the relationshipbetween the x axis of the Y axis and the y and z axes is simply checked,it can be estimated whether the person being measured 2 walksempty-handed or with a specific article 8 to be carried. In thisapplication example, therefore, the Y-axis direction is checked first.

When the Y axis is directed to a direction different from the z-axisdirection, it is possible to estimate an approximate moving state of theperson being measured 2 only from the detected raw signal WY(t) in theY-axis direction obtained from the three-axis acceleration sensor 72.

However, when the person being measured 2 moves (walks) with the article8 as shown in (a) of FIG. 25 , the article 8 slightly moves vertically.Therefore, the movement components of the vertical movement of thearticle 8 are mixed as disturbance noise into the detected raw signalsWX(t) and WY(t) collected from the three-axis acceleration sensor 72.

The disturbance noise component Wz(t) caused by the vertical movement ofthe article 8 to be carried appears most significantly in the detectedraw signal WX(t) collected from the three-axis acceleration sensor 72.It is thus possible to reduce the disturbance noise components mixedinto WX(t) and WY(t) using the disturbance noise component Wz(t)appearing in WX(t).

In the system (signal processing system) according to the presentembodiment, a plurality of detected raw signals that change over timeare collected from one or more sensors, and the detected raw signals (orsome of them) are processed to reduce noise and extract a specificsignal component. As an example of the extraction of this specificsignal component, a method for extracting a predetermined criterion frompredetermined frequency components Q41L, Q42L and Q43L, which areobtained after the low-pass filtering process 138 shown in FIG. 3 ,using the equations (1) to (3), has been described. However, the presentembodiment is not limited to this example. In the present embodiment,the specific signal component can be extracted using any otherprocessing methods. As an example of signal processing to be performedat this time, the process using the equation (4) or (6) shown in (c) ofFIG. 25 can be performed. However, the present embodiment is not limitedto this signal processing. For example, another optional signalprocessing method such as vector operation shown in (d) of FIG. 25 canbe adopted.

The detected signal generation (signal processing) 96 (FIG. 12 or 13 )after reduction of disturbance noise component Wz(t) due to the verticalmovement of the article 8 to be carried, brings about the advantagecapable of high-accuracy behavior estimation. When the drive device 44is driven to provide service to a user based upon the behaviorestimation result 16, the service can be provided more accurately.

As a further application example, high-accuracy detection/behaviorestimation/service can be provided by combining the detected raw signalsobtained from the sensors. As a method for combining the detected rawsignals obtained from the sensors, the detected raw signals collectedfrom the three-axis acceleration sensor 72 shown in FIG. 12 or 13 can becombined with the position detection function 98. In other words, therelative position of the wristband-shaped activity meter (sensor device)4 can be estimated by subjecting the detected raw signals collected fromthe three-axis acceleration sensor 72 to signal processing (signaloperation). Feeding the result of the position detection function 98described above back to this estimation result improves the accuracy ofposition detection of the wristband-shaped activity meter (sensordevice) 4.

As another application example, a method for improving the detectionaccuracy by combining the detected raw signals collected from thewristband-shaped activity meter (sensor device) 4 shown in (a) of FIG. 1and the detected raw signals collected from the sensor device 6 fixed toa predetermined object such as the dolly 122 is shown in FIG. 26 . Ashas been described, the sensor device 6 also includes the three-axisacceleration sensor 72 and the three-axis geomagnetism sensor 74 and hasthe position detection function 98 using the communication control unit84.

Thus, a gravity direction (criterion B) 104 is extracted in thewristband-shaped activity meter (sensor device) 4 (Step 1) and thesensor device 6 (Step 2) at the same time. The sensor device 6 may befixed to part of the dolly 122 as shown in FIG. 16 and may be fixeddirectly to the article 8 itself along FIG. 25 .

Then, based upon the angle θyz of each extracted gravity direction(criterion B) 104, the detected raw signals are processed in thewristband-shaped activity meter (sensor device) 4 (Step 3) and thesensor device 6 (Step 4) at the same time to reduce noise components.

After that, based upon a result of the above reduction of noisecomponents, the behavior estimation engine 56 of the edge device 42determines behavior estimation for the wristband-shaped activity meter(sensor device) 4 (Step 5) and the sensor device 6 (Step 6).

In Step 7, it is determined whether high-accuracy behavior estimation isnecessary. If high-accuracy behavior estimation is not necessary, onlythe behavior estimation results based on data obtained from only thewristband-shaped activity meter (sensor device) 4 are used (Step 8) topropose business improvement and provide service based on the behaviorhistory of the person being measured 2 (Step 11) or terminate theprocess (Step 14).

If high-accuracy behavior estimation is necessary to the contrary as aresult of the determination in Step 7, a behavior estimationdetermination result from the wristband-shaped activity meter (sensordevice) 4 and that from the sensor device 6 are combined (Step 9).

Based on the determination result in Step 9, the business improvementproposal and service providing (Step 11) based on the activity historyof each person being measured 2 or process termination (Step 14) isselected.

Similar to Step B9 in FIG. 22B, the process from Step 1/Step 2 to Step11 corresponding to one cycle period is repeated until the behaviorestimation process or service providing process is completed. Thedetermination as to whether to repeat the process is made in Step 12.When the behavior estimation process or business improvementproposal/service providing process is completed in Step 12, a series ofprocesses is terminated (Step 13).

As described above, the estimated behavior results from only thedetected raw signals obtained from the wristband-shaped activity meter(sensor device) 4 and the behavior estimation results obtained from onlythe detected raw signals obtained from the sensor device 6 are combinedwith each other. This combination brings about the advantage capable ofhigh-accuracy behavior estimation and high-quality service providing.

The method for collecting the detected raw signals during the behaviorof the person being measured 2 to estimate the behavior of the personbeing measured 2 has been described so far. However, the presentembodiment is not limited to this method. In the system (behaviorestimating system) according to the present embodiment, the state of atarget object can be estimated from the detected raw signals collectedfrom a predetermined object (other than human beings and animals) andits corresponding service can be provided.

An application example (FIG. 27 ) of estimating the state of an elevatorcar 202 itself (and service providing as needed) using the raw signalsWY(t) and WZ(t) detected from the sensor (e.g., three-axis accelerationsensor 72) fixed to the elevator car 202, will be described.

If the three-axis acceleration sensor 72 is inclined and fixed to afixing base 206 of the elevator car 202, the direction of the Z axis setin the three-axis acceleration sensor 72 is inclined to the z axis alongthe criterion B (Gravity) 104. Then, the rolling components 208 in they-axis direction, generated when the elevator car 202 moves verticallyin the z-axis direction, are mixed as noise components into the detectedraw signals WZ(t) and WY(t).

In this case, too, the components Q41L and Q42L are extracted throughthe low-pass filtering process 138 (FIG. 3 ) to allow an inclinationangle θyz for attaching the three-axis acceleration sensor 72 to becalculated using the equation (1).

Furthermore, using a result of the above, the signals are separated intoWy(t) corresponding to the rolling component 208 and Wz(t) (from whichthe mixed rolling component 208 is reduced) representing only thevertical movement of the elevator car 202.

Using the calculation result Wz(t), the estimation engine 56 of the edgedevice 42 determines whether the elevator car 202 moves vertically asscheduled. If an unexpected detected signal Wz(t) is obtained during thevertical movement, the elevator is regarded as malfunctioning, andservice for repairing the elevator is provided.

Simultaneously with the above, the estimation engine 56 of the edgedevice 42 monitors the state of the rolling 208 generated during thevertical movement of the elevator car 202 from the detected signalWy(t). If abnormal rolling is found during the vertical movement of theelevator car 202, service for repairing the elevator is provided.

Using the system (behavior estimating system 52) according to thepresent embodiment as described above brings about the advantage thatdifferent detected signals Wz(t) and Wy(t) can be monitoredsimultaneously and accurately.

The foregoing descriptions have been so far made with an emphasis on theacceleration sensor as a sensor. However, the sensor is not limited tothe acceleration sensor. In the present embodiment, an angular velocitysensor can be used. FIG. 28 shows an application example in which thepresent embodiment is applied to a bus, a truck or an automobile 210.Criterion B (gravity direction) 104 is extracted from the detected rawsignals WX(t), WY(t) and WZ(t) collected from the three-axisacceleration sensor 72 fixed in the bus, truck or automobile 210. Basedon the criterion B, acceleration components Wx(t), Wy(t) and Wz(t) inthe z-axis, y-axis and x-axis directions are extracted by rotationcoordinate transformation or vector synthesis to make it possible todetect the states of vertical vibration, acceleration and deceleration,and right- and left-direction course change of the bus, truck andautomobile 210. In the application example shown in FIG. 28 , theacceleration/deceleration direction of the bus, truck or automobile 210indicates the y-axis direction, and the z-axis direction coincides withthe gravity direction 104 corresponding to the criterion B. Though notshown, the direction in which the bus, truck or automobile 210 turnssideways (direction of course change) is directed to the x-axisdirection.

The present embodiment is not limited to the above. The three-axisangular velocity sensor 212 can be fixed to the steering wheel 214 ofthe bus, truck or automobile 210 by means of the fixing member 50 (e.g.,adhesive) to extract a history of the wheel operation during thedriving. In this case, however, the vertical vibration, acceleration anddeceleration, and right- and left-direction course change of the bus,truck and automobile 210 are mixed as disturbance noise into therotation direction 218 of the steering wheel detected by the three-axisangular velocity sensor 212.

To prevent the disturbance noise, disturbance noise components can bereduced by processing the raw signals detected from the above three-axisangular velocity sensor 212 using the raw signals detected from thethree-axis acceleration sensor 72. Removing the disturbance noisecomponents from the three-axis angular velocity sensor 212 using the rawsignals detected from the three-axis acceleration sensor 72 brings aboutthe advantage capable of detecting the operation history of the steeringwheel 214 with accuracy, estimating the behavior of the driver andproviding the driver or passengers with appropriate service.

In the descriptions given so far, the fixed physical quantity such asthe geomagnetism direction and gravity direction is taken as an exampleof criteria 102 and 104. However, the criteria are not limited to thephysical quantity. In the system (signal processing system) according tothe present embodiment, for example, a prescribed route direction (e.g.,the direction of a slope) may be set to criterion C (prescribed route)106. This application example will be described with reference to FIG.29B.

In the embodiments described so far, the floor surface 128 needs to bein a direction perpendicular to the gravity direction as a prerequisitecondition on which the criterion B (gravity) 104 can be used. However,when the person being measured 2 moves along a slope 220 with apredetermined inclination of angle μ with respect to the planeperpendicular to the gravity direction 104 as shown in FIG. 29A, if thedirection along this slope is set as a new criterion C (prescribedroute) 106, the behavior of the person being measured 2 can be estimatedwith efficiency.

While the direction (the direction in which the person being measured 2moves) along the slope with an inclination of angle μ with respect tothe floor surface 128 having a plane perpendicular to the gravitydirection 104 is set as a new criterion C (prescribed route) 106, thedirection will be defined as a u axis and a direction orthogonal to thedirection (a direction along the slope 220) will be defined as a w axis.

As a method for setting the direction along the slope 220 to thecriterion C (prescribed route) 106, the movement routes of a pluralityof persons being measured 2 are accumulated, and based on a result ofthe statistical processing, (the direction of) the criterion C(prescribed route) 106 is defined/extracted in the present embodiment.

The position detecting signal generation unit 46 (A) to (C) is providedwithin a range in which the person being measured 2 moves in advance asdescribed with reference to FIGS. 12 and 13 . The position detectingsignal generation unit 46 (A) to (C) continues to transmit timeinformation at the time of transmission using wireless communication.The communication control unit 84 of the wristband-shaped activity meter(sensor device) 4 receives the time information at the time oftransmission, detects a time lag between the transmission and reception,and detects position information of the communication control unit 84using trigonometry.

When the person being measured 2 climbs the slope 220, historyinformation about a change in position of the wristband-shaped activitymeter (sensor device) 4 is accumulated (in the memory unit 82 of FIGS.12 and 13 ). From the accumulated history information, the criterion C(prescribed route) 106 can be extracted.

In FIG. 29A, the history information on the movement route obtained whenthe person being measured 2 climbs the slope 220 and pushes the dolly122 is used. However, the present embodiment is not limited to thehistory information. As shown in FIG. 29B, a movement route historyobtained when the persons being measured 2-1 to 2-3 each equipped withthe wristband-shaped activity meter (sensor device) 4 move empty-handedon the slope 220, may also be used.

A method for extracting the direction of criterion C (prescribed route)106 will be described with reference to FIG. 29C. In FIG. 29C, criterionC (prescribed route) 106 is extracted utilizing a history of a change inposition during the movement of N persons being measured 2.

First, “n=0” is set as the initial value (Step 21), and the movementroute of the n-th person being measured 2 is extracted from the historyinformation of a change in position of the activity meter 4 attached tothe n-th person being measured (Step 23). In each cycle of this process,the value of n is incremented by one in Step 22. Then, the extraction ofthe movement route history is repeated until the value of n exceeds thevalue of N (Step 24) to sequentially accumulate the movement routes of Npersons being measured (history information of a change in position).

After the collection of a series of data is completed, the movementroutes of N persons being measured are analyzed statistically (Step 25).Using the results, a no-entry area of the persons being measured isextracted in Step 26. As shown in FIGS. 29A and 29B, the person beingmeasured 2 who climbs the slope 220 cannot pass under the slope(position lower than the floor surface 128). Thus, the underside of theslope 220 (in the ground of the slope) shown in FIGS. 29A and 29Bcorresponds to the no-entry area. Using no-entry area information of theperson being measured so calculated, the inclination angle μ of theslope 220 is calculated (Step 27).

Thus, the prescribed route (criterion C (prescribed route) 106) servingas a criterion, which the person being measured 2 has passed, iscalculated from the past history information (Step 28).

The present embodiment and its application examples described so far aredirected to a method for utilizing detected raw signals collected fromthe sensors related to the acceleration, angular velocity and magneticfield. However, they are not limited to this method. The raw signalsdetected from any type of sensor can be used. Another applicationexample of using photoelectric conversion signals detected from anoptical sensor is shown in FIG. 30 .

As a criterion example based on the photoelectric signal of the opticalsensor, a sunlight direction can be set to criterion D_108. For example,in the outdoors on a clear sky day, criterion D (sunlight direction) 108can be extracted using sunlight from the sun 230.

As an example of providing service to a user using the criterion D(sunlight direction) 108, a method for detecting a backlight stateautomatically is shown in FIG. 30 . In other words, it is automaticallydetected whether the photography of a subject 232 using a photographycamera 252 is in a backlight state to optimize the photography state ofthe subject 232.

When the sunlight direction (criterion D) 108 is extracted in theoutdoors on a clear sky day, an airplane (noise source) 234 and ahelicopter (noise source) 236 flying in the sky are noise sources thatblock the sunlight from the sun 230. As a noise source on the ground,light emitting objects such as car lights (noise sources) 246 that aremounted on automobiles 238 are present everywhere on the ground. Thesenoise sources may adversely influence the extraction of the criterion D(sunlight direction) 108.

Even though sunlight is temporarily blocked under the influence of theairplane (noise source) 234 and helicopter (noise source) 236 flying inthe sky, the blocked state does not continue for a long period of time.If, therefore, low-frequency components are extracted from the detectedlight of sunlight emitted from the sun 230 (e.g., utilization of thelow-pass filtering function 138 in FIG. 3 ), the adverse influence fromthe airplane (noise source) 234 and helicopter (noise source) 236 can beeliminated.

As shown in FIG. 30 , the light-receiving surfaces of the opticalsensors (solar cell panels 262, 264 and 266) placed in the photographycamera 252 are non-parallel to one another. It is thus possible tocalculate the sunlight direction (criterion D) 108 using a difference inthe number of photoelectric signals detected by the optical sensors(solar cell panels) 262, 264 and 266. The optical sensors (solar cellpanels 262, 264 and 266) can independently be placed in positions awayfrom the photography camera 252.

Since the sun 230 is present in the sky when the photography time of thesubject 232 is around noon, the criterion D (sunlight direction) 108 issubstantially perpendicular (sky direction) to the ground. In contrast,the light emitted from light-emitting sources on the ground, such as carlights (noise sources) 246 that are mounted on the automobiles 238 areirradiated at an angle close to the extension direction of the ground.

Noise components can be removed by using a difference in angle betweenthe light emitted from the light source on the ground and the sunlightdirection (criterion D) 108. Specifically, the criterion D (sunlightdirection) 108 is calculated with accuracy using, e.g., the low-passfiltering process 138 in FIG. 3 (extraction of components whosefrequency is not higher than cutoff frequency fc), and the detected rawsignals WX(t), WY(t) and WZ(t) collected from the optical sensors (solarcell panels) 262, 264 and 266, respectively are processed using thecriterion D, thus calculating the intensity of the sunlight whose noisecomponents are reduced.

As a means for providing service to a photographer 250 when backlight isdetected, an aperture 256 (a type of drive device for exposureadjustment) and a light 258 (drive device) of the photography camera 252can be controlled to take measures against the backlight.

An example of this series of processing method will be described indetail with reference to FIG. 31 . The start process in Step 31corresponds to the pre-processing start immediately before thephotographer 250 takes a picture of the subject 232 with the photographycamera 252.

Prior to the actual picture taking, in Step 32, the detected raw signalscollected from the optical sensors (solar cell panels) 262, 264 and 266having light-receiving surfaces perpendicular to the X-axis, Y-axis andZ-axis directions which are orthogonal to one another are subjected to alow-pass filtering process (signal components whose frequency is nothigher than cutoff frequency fc are extracted to reduce the influence oflight blocked by the airplane 234 and helicopter 236 which are noisesources).

Then, the sunlight direction (criterion D) 108 is extracted (Step 33)based upon a result of the above. In the extraction of the sunlightdirection (criterion D) 108 (Step 33), the equation (1) obtained fromFIG. 2 is used.

In Step 34, the extracted sunlight direction (criterion D) 108 iscompared with the direction to which the photography camera 252 isdirected to determine whether the subject 232 is in the backlight state.If the subject 232 is not in the backlight state, the camera starts totake a picture in the state as shown in Step 34.

If the subject 232 is in the backlight state, the following measures aretaken in accordance with the intensity of the sunlight as an example ofservice to be provided to the photographer 250.

As shown in FIG. 30 , there are a number of light emitting sources(noise sources) on the ground. To calculate the intensity of thesunlight, therefore, it is necessary to take measures to reduce noisefrom the light intensity detected in Steps 35 to 37.

The light intensity (detected raw signals WX(t), WY(t) and WZ(t))detected from the optical sensors (solar cell panels) 262, 264 and 266are subjected to low-pass filtering process 138 to extract predeterminedfrequency components Q41L, Q42L and Q43L. Based on this result, theangle θyz of the criterion D (sunlight direction) 108 can be calculatedand the irradiated light intensity can be calculated by arithmeticprocessing using the equation (4) or (6) (Step 35).

Since the position of the sun 230 can be predicted from thepicture-taking time information, the angle between the ground and thecriterion D (sunlight direction) 108 can also be predicted. From theprediction result, the direction of the light emitting source (noisesource) on the ground can be estimated, and noise components mixed fromthe light emitting source (noise source) can be calculated (Step 36) asin the arithmetic processing using the equation (4) or (6).

Since the criterion D (sunlight direction) 108 is not alwaysperpendicular to the ground, the noise components from the lightemitting source (noise source) on the ground are mixed into theirradiated light of the sunlight direction (criterion D) componentscalculated in Step 35. Signal processing (arithmetic processing betweensignals) is performed using the noise components calculated in Step 36to allow noise to be reduced further (Step 37).

The backlight correction method that is a method for providing serviceto the photographer 250 in the example of FIG. 30 includes two methodsof adjustment of the aperture 256 and emission of the light 258 toselect the service providing method appropriately according to theintensity of sunlight (amount of sunlight). The selection of the serviceproviding method according to the amount of sunlight is performed inStep 38.

More specifically, when the amount of sunlight is small (the backlightstate is not so strong), the size of the aperture 256 (drive device)with an exposure amount adjustment function is changed (Step 39). Thus,the light-receiving amount of the photography camera 252 isautomatically controlled to correct the exposure light. As anothermethod for correcting an exposure amount in this case, the gain of asignal detected from an imaging device (not shown) which is built in thephotography camera 252 can automatically be changed.

On the other hand, since the backlight state is strong when the amountof sunlight is very large, the exposure light is not correctedsufficiently within the adjustment range of the aperture 256 (exposureamount adjustment using the drive device). In this case, picture takingis started (Step 41) while the light (drive device) 258 attached to thephotography camera 252 is lit to irradiate the subject 232 and takemeasures against the backlight as shown in Step 40. When the picturetaking is completed, an end process (Step 42) is performed.

A method for reducing noise from the detected signals and providingservice using only the photoelectric signals obtained from the opticalsensors (solar cell panels) 262, 264 and 266 has been described withreference to FIGS. 30 and 31 . However, the present embodiment is notlimited to this method, but the photoelectric signals can be combinedwith other types of signals detected from other sensors or combined withanother criterion. This brings about the advantages capable of improvingthe accuracy of the detected signals and the accuracy of the service tobe provided.

A specific example of the above will be described below. The sunlightdirection (criterion D) 108 varies depending upon picture-taking timeand picture-taking locations. As has already been described withreference to FIG. 12 or 13 , the three-axis geomagnetism sensor 74 inthe behavior estimating system 52 can detect the direction of theearth's axis. Also, the picture-taking time can be confirmed accuratelyusing time information transmitted from the position detecting signalgeneration unit 46. It is thus possible to calculate the sunlightdirection (criterion D) 108 from the earth's axis direction and thepicture-taking time information. With this information, the sunlightdirection (criterion D) 108 calculated from the optical sensors (solarcell panels) 262, 264 and 266 can be corrected to calculate the sunlightdirection (criterion D) 108 with high accuracy and obtain an amount ofsunlight with higher accuracy. As a result, a more accurate service canbe provided to the photographer 250.

As an application example of the present embodiment, a method (noisereduction method) for improving the accuracy of an audio signal detectedby combining directional microphones 282, 284 and 286 will be describedwith reference to FIG. 32 . Consider the case where only the audiosignal output from a specific speaker 270 set in a fixed position isdetected with high accuracy using the directional microphone 282.

There are a large number of disturbance audio noise sources 272 and 274around the specific speaker 270 set in the fixed position, and thedisturbance noise components generated therefrom are mixed into thedirectional microphone 282. In this case, a fixed speaker direction 110is set to criterion E (Y-axis direction), and the directional microphone282 to direct a signal is directed to the Y-axis direction. Then, thedisturbance audio noise components mixed from the directions other thanthe criterion E (direction of the fixed speaker) are detected by thedirectional microphones 284 and 286 that are directed to the X-axisdirection and the Z-axis direction, respectively.

Then, the signal processing using the detected raw signals collectedfrom the directional microphones 284 and 286 is performed to reduce thenoise components mixed into the directional microphone 282.

If the value of cutoff frequency fc in the low-pass filtering process138 shown in FIG. 3 is set to 50 Hz, 60 Hz or 0.5 Hz to filter the audiosignals (detected raw signals), the following disadvantages will occur.

The audible range of human beings in the frequency characteristics ofaudio signals contains a number of frequency components the frequency ofwhich is higher than 60 Hz, as shown in FIG. 33 . If, therefore, theaudio signals (detected raw signals collected from the directionalmicrophone 282) are directly subjected to the low-pass filtering process138 (FIG. 3 ) to extract only the low-frequency components of 60 Hz orlower, there is a risk that important audio signal components will beerased.

As a means for solving the above problem, an audio signal (detected rawsignal 290) can be Fourier transformed 292 and a low-pass filteringprocess 294 can be performed for the amplitude of each of the Fouriertransformed frequency components, as shown in FIG. 34 . As described inFIG. 12 or 13 , the detected raw signals collected from the directionalmicrophone 282 (signals converted into digital signals by the A/Dconverter 78) are stored temporarily in the memory unit 82 andprocessed. Thus, the low-pass filtering process can easily be performedfor the amplitude of each of the Fourier transformed frequencycomponents (Fourier coefficients).

However, the present embodiment is not limited to the above. The memoryunit 82 can properly be utilized in the order described as follows: “Thedetected raw signals are Fourier transformed every predeterminedperiod”, “the Fourier coefficients are stored in sequence in the memoryunit 82”, “the result of the low-pass filtering process for each of theFourier coefficients is stored again in the memory unit 82” and “thesignals are processed using information stored in the memory Unit 82”.

Therefore, as an example of the processing method of the Fouriertransformation 292 described with reference to FIG. 34 , the audiosignals (detected raw signals 290) stored appropriately in the memoryunit 82 can be read in sequence to store a new additional audio signal(detected raw signal 290) in the memory unit 82 and process the audiosignals (detected raw signals 290) stored in the memory unit 82 inparallel at the same time.

A method for inserting the Fourier transformation 292 has been describedwith reference to FIG. 34 , taking an audio signal as an example.However, the present embodiment is not limited to the audio signal, butthe Fourier transformation 292 can be performed for all of the detectedraw signals 290 and the low-pass filtering process can be performed withrespect to a change in amplitude of the frequency component.

The tendency toward the processes performed according to the presentembodiment and using its application examples described so far, will bedescribed with reference to FIGS. 35 to 37 . In the present embodimentand its application examples, predetermined information is extractedusing the raw signals detected from different sensors. As one type ofthe extracted predetermined information, a predetermined criterion canbe used. Consider a case where a disturbance signal (including thelow-frequency components) is added to the extraction of a criterion inStep 51 prior to the extraction of the predetermined criterion. If thepredetermined criterion is extracted in this state (Step 52), an erroris easily mixed into the extracted criterion (Step 62).

If, furthermore, service is provided using the result of the estimationprocess in Step 54 (Step 55), a difference in the contents of serviceproviding, which is caused by the mixture of an error into thecriterion, is likely to occur (Step 65).

In the present embodiment and its application examples, the above cycleis repeated until the estimation process or service providing process iscompleted, as shown in Step 56.

Determining the states of a variation in results of estimation (Step 64)and a difference in contents of service providing (Step 65) caused bythe error of criterion generated when the above cycle is repeated with adisturbance signal containing low-frequency components added (Step 51),it is possible to determine whether the technologies of the presentembodiment and its application examples have been adopted.

An example of a method for adding a disturbance signal containinglow-frequency components to the raw signals detected from differentsensors (Step 51) will be described below.

As a method for adding a disturbance signal containing low-frequencycomponents to the raw signals (acceleration waveforms) detected from thethree-axis acceleration sensor 72, for example, an apparentgravitational acceleration can be given.

For example, if the detected raw signals (acceleration waveforms) arecollected from the three-axis acceleration sensor 72 with the elevatorcar 202 moving vertically using a rope 204 as shown in FIG. 36A, thevalue of the apparent gravitational acceleration G varies. If the angleθyz is calculated using the foregoing equation (2) or (3) as a result,an error occurs in the value of the calculated angle θyz.

Furthermore, when the door of the elevator car 202 is opened to move alarge dolly 302 on which the dolly 122 is put, the apparentgravitational acceleration component is superimposed in a directionparallel to the floor surface 128. In this case, even though theequation (1) in which the value of the gravitational acceleration G isnot used is used, an error is mixed into the value of the angle θyzcalculated from the raw signals (acceleration waveforms) detected fromthe three-axis acceleration sensor 72 of the wristband-shaped activitymeter (sensor device 4) attached to the person being measured 2.

The present embodiment is not limited to the above. As shown in (a) ofFIG. 36B, a disturbance signal containing low-frequency components canbe added using a slope (Step 51 in FIG. 35 ). For example, even thoughthe behavior of the person being measured 2 is estimated using thewristband-shaped activity meter (sensor device 4) when the person beingmeasured 2 applies force to the dolly 122, which is stopped in advanceby a fixed stopper 312, to start to move it, a variation in contents ofthe behavior estimation (Step 64 in FIG. 35 ) and a difference incontents of service providing (Step 65 in FIG. 35 ) will be caused.

Similarly, as shown in (b) of FIG. 36B, a similar symptom appears evenwhen the person being measured 2 starts to move the dolly 122 after heor she releases a vertical moving stopper 314 to stop the dolly 122 on aslope. It can be seen from a variation in results of the estimation(Step 64 in FIG. 35 ) and a difference in service to be provided (Step65 in FIG. 35 ) that the technologies of the present embodiment and itsapplication examples are adopted.

On the other hand, as a method for adding a disturbance signalcontaining low-frequency components to the three-axis geomagnetismsensor 74 (Step 51 in FIG. 35 ), a permanent magnet 304 can be stuck onthe surface of the wristband-shaped activity meter 4 to add adisturbance in the geomagnetism direction, as shown in (a) of FIG. 37C.

Furthermore, as a method for adding a disturbance signal containinglow-frequency components to the photoelectric signals obtained from theoptical sensors (solar cell panels) 262, 264 and 266 (Step 51 in FIG. 35), a light 306 serving as a light emitting source can be placed in afixed light stand 308 to disturb the sunlight direction.

A method using the expansion/contraction matching technique for thebehavior estimation and state estimation in the present embodiment orits application examples has been described. As an example of theexpansion/contraction matching technique, the DP matching (dynamicprogramming matching) has been described with reference to FIGS. 18 to20 . The method shown in FIG. 37 makes it possible to discover whetheror not the expansion/contraction matching technique was used at the timeof the estimation process (behavior estimation or state estimation).

The expansion/contraction matching technique has the feature that asimilarity among patterns can be calculated while considering partialexpansion/contraction in the patterns. From different sensors used inthe system (signal processing system) according to the presentembodiment or its application examples, the value of the detected rawsignals may often vary over time. Especially in this case, when themovement speed of the person being measured 2 and the state change speed(e.g., moving speed) of a predetermined object such as the dolly 122 arechanged, partial expansion/contraction (expansion/contraction in thetime axis direction) is generated in the variation in the value of thedetected raw signals regarding a lapse of time.

If, therefore, the estimation process (behavior estimation or stateestimation) is performed using the foregoing expansion/contractionmatching technique when partial extension/contraction occurs (in thetime axis direction) in the detected raw signals collected from thedifferent sensors in accordance with the moving speed of the personbeing measured 2 and the state change speed of a predetermined object inthe system (signal processing system) according to the presentembodiment or its application examples, the advantage of improving theaccuracy of the estimation is brought about. In other words, even thoughthe movement of the person being measured 2 and the state change speedof the predetermined object vary temporarily, the temporal change inprocessing speed is absorbed at a stage of the estimation process(behavior estimation or state estimation).

For example, as a method for varying the behavior speed of the personbeing measured 2, the person being measured 2 is caused to move muchmore slowly or much more quickly than standard as shown in Step 71.

As a method for changing the processing speed of a specific object, thespeed (processing speed) at which an object (e.g., the dolly 122) towhich the sensor device 6 is fixed can be changed more greatly thanstandard as shown in Step 72. As a specific method, the dolly 122 can bemoved quickly (high-speed processing) or slowly (low-speed processing).

Then, the raw signals detected from the different sensors are collected(Step 73) and at least some of the detected raw signals (predeterminedsignal components and predetermined frequency components or sensorselection) are processed (Step 75).

Prior to the signal processing, predetermined information can beextracted from at least some of the detected raw signals (predeterminedsignal components and predetermined frequency components or sensorselection) and the signal processing can be performed based upon aresult of the detected information (Step 75). The predetermined criteria102 to 110 can be caused to correspond to the predetermined information(Step 74).

After that, if the extension/contraction matching technique is employedwhen the estimation process (behavior estimation or state estimation) isperformed (Step 76), there is a tendency that the results of theestimation process are the same even though the moving speed of theperson being measured and the processing speed vary greatly (Step 86).

Furthermore, when the service providing process is performed based upona result of the process performed in Step 76 (Step 77), there is atendency that the results of the service providing are the same eventhough the moving speed of the person being measured and the processingspeed vary greatly (Step 87).

If, therefore, a tendency that the result of the estimation process orthat of the service providing does not vary when the speed is changedcan be found (Step 86 and Step 87), it can be understood why theextension/contraction matching technique is used for the estimationprocess (Step 88).

Moreover, the foregoing cycle is repeated until the estimation processor the service providing is terminated as shown in Step 78. If the cycleis so repeated, the number of samples to check the result of theestimation process and that of the service providing for each cycle isincreased. If the extension/contraction matching technique is used, theprobability that the determination can be made is increased.

In the system (signal processing system) according to the presentembodiment shown in FIGS. 12 and 13 , the drive device 44 is set toallow service to be provided to a user based upon a result of each ofthe different estimation processes. As one example of the drive device44, FIG. 38 shows the glasses-type wearable terminal 1100. Part of theglasses-type wearable terminal 1100 may have a function of the sensordevice. In other words, the glasses-type wearable terminal 1100 mayinclude a camera and a microphone or have a vibration detection functionto detect a predetermined instruction (control information) input from aperson equipped with the terminal (person being measured 2). As theinstruction input from the person, for example, he or she can block alens unit of the camera, clap his or her hands for the microphone,request the next display by voice, provide the vibration detectionfunction with a predetermined vibration, etc. If the glasses-typewearable terminal 1100 that originally has a function of the drivedevice 44 has a function of the sensor device, the advantage that theperson equipped with the terminal (worker or person being measured 2)can input data without using his or her hands is brought about.Furthermore, the person equipped with the terminal (worker or personbeing measured 2) may estimate his or her behavior or state with thebehavior estimation engine 56 (FIG. 12 or 13 ) based upon the inputdata.

The glasses-type wearable terminal 1100 includes a projection unit(display information generation unit) 1102, a screen (optical pathsynthesis unit) 1106, a drive unit (which may also be referred to as animage display circuit, a light source drive circuit and a signalprocessing unit) 1134, a wireless communication unit 1136 and the likeand is operated by power applied from a power supply unit 1132 that is,for example, a button cell.

The wireless communication unit 1136 built in the glasses-type wearableterminal 1100 performs information communication between the edge device42 and the server 40 connected to an external network. The informationcommunicated from the edge device 42 is displayed to the person equippedwith the terminal (worker or person being measured 2) through theprojection unit 1102.

The projection unit 1102 includes a light source unit 1104, an imagedisplay unit 1110, a half mirror surface 1112, a total reflectionsurface 1114, a light emission surface 1116, a lens group 1120 and thelike. The information communicated from the edge device 42 is displayedon the image display unit 1110 of the projection unit 1102 as videoinformation or image information. Then, non-parallel light (light withdivergence properties: referred to as diverging light) 1108 which isemitted from the light source unit 1104 of the projection unit 1102 isapplied to the image display unit 1110 via the half mirror surface 1112,and the light reflected by the unit 1110 is projected onto the screen1106 via the lens group 1120.

The screen 1106 includes a front-side transparent refractive body 1124,a Fresnel lens type half mirror surface 1122 and a back-side transparentrefractive body 1126. Part of the diverging light 1108 is reflected bythe Fresnel lens type half mirror surface 1122 and arrives at the eyesof the person equipped with the terminal (worker or person beingmeasured 2) through the front-side transparent refractive body 1124 toform a virtual image corresponding to the image displayed on the imagedisplay unit 1110.

As the light source unit 1104, a dimmer type white LED light source canbe used. The dimmer type white LED light source includes a plurality oflight emitting diodes (LED) which adjust the output light amountsindependently and whose emission colors are different from one another.This brings about the advantage capable of displaying the optimumemission color corresponding to the usage environment to the personequipped with the terminal (worker or person being measured 2). Forexample, when the glasses-type wearable terminal 1100 is used in a cleanroom of lighting environment the color of which is mainly orange, animage can be displayed in combination of colors excluding the orangecolor. If, furthermore, an image is displayed in the color that can beeasily recognized by the person equipped with the terminal (worker orperson being measured 2), the advantage that he or she can avoid eyefatigue and its related migraine is brought about.

For example, a reflective liquid crystal display (LCD) module is usedfor the image display unit 1110, and an image displayed thereon iscontrolled by the drive unit 1134. The drive unit also controls anemission state of the light source unit 1104.

FIG. 39A shows a procedure in which a worker equipped with theglasses-type wearable terminal 1100 performs an operation in response toan operation instruction from the edge device 42. As a specific exampleof the operation, the worker may inspect a manufacturing apparatus,repair a malfunctioning machine, and the like.

For example, when a worker equipped with the glasses-type wearableterminal 1100 arrives at a workplace, he or she pushes an work startbutton and make a special gesture. A detection signal from apredetermined sensor device inserted into the work start button istransmitted to the edge device 42. Alternatively, an imaging device thatmonitors the behavior of the worker may determine the gestureautomatically and transmit a result of the determination to the edgedevice 42.

The contents of the work are divided in advance into a plurality of workunits (a plurality of divided works). When the completion of a work ofone unit is detected (Step SA5), the next work instruction (Step SA6) isdisplayed on the glasses-type wearable terminal 1100.

FIG. 39B shows an example of another operation to be performed by asystem using the glasses-type wearable terminal 1100 according to thepresent embodiment. FIG. 39B shows a control operation to be performedafter the worker has moved to the workplace.

The glasses-type wearable terminal 1100 has the position detectionfunction 98 shown in FIG. 12 or 13 . The edge device 42 detects aposition of the worker equipped with the glasses-type wearable terminal1100 using the position detection function 98 (Step SC1). When the edgedevice 42 supplies a movement instruction to the worker through theglasses-type wearable terminal 1100 (Step SC2), the worker starts tomove.

When the stoppage of the worker is detected (Step SC3), it is determinedwhether the worker stops in a normal position (an instructed targetposition) (Step SC6). If the worker does not stop after a while, it isdetermined whether time elapses a given period of time or longer (StepSC4). If the worker does not stop when time elapses a given period oftime or longer, it is determined that any trouble has occurred, and thealarm is given through the glasses-type wearable terminal 1100 toinstruct the worker to stop (Step SC5).

When the worker does not stop in the normal position in Step SC6, it isdetermined that the work position is an abnormal position (Step SC7),and the alarm is given through the glasses-type wearable terminal 1100(Step SC8) to instruct the worker to move to a predetermined position.

On the other hand, when the worker stops in the normal position, thework instruction described with reference to FIG. 39A is started.

FIG. 40 shows an example of a work process to be performed by a workeraccording to the foregoing work instruction. Consider here that theworker equipped with the glasses-type wearable terminal 1100 arrives ata workplace and pushes a work start button in the workplace (or makes aspecial gesture). When the work start button is pushed, communicationsare carried out between the glasses-type wearable terminal 1100 and theedge device 42. Assume here that the work of the worker in the workplaceis to tighten a screw 2001 in a housing 2005 of a manufacturingapparatus. As the housing before the start of the work, the top 2006 ofthe housing 2005 is opened and the opening can be seen.

In accordance with the contents of the instruction transmitted from theedge device 42, a message such as “TIGHTEN THE SCREW” is displayed onthe glasses-type wearable terminal 1100 (Step SB1). In accordance withthe contents of the instruction, the worker inserts a driver 2002 in thehousing from the opening thereof and starts to tighten a screw 2001.

A sensor (e.g., angular velocity sensor) 2021 attached to the screw 2001or the driver 2002 allows the angular velocity to be detected (StepSB2). Thus, when the screw tightening work is started, the angularvelocity sensor 2021 detects the rotation of the screw.

When the rotation detection signal is transmitted to the edge device 42,the start of the work is recognized. When the start of the work isrecognized, the edge device 42 outputs an instruction to erase thecurrent message “TIGHTEN THE SCREW”.

When the tightening of the screw 2001 is completed, the detection outputof the angular velocity sensor 2021 becomes zero. Upon receipt of thesensor detection signal, the edge device 42 determines the “completionof tightening of the screw” (Step SB3).

Immediately after that, the edge device 42 transmits the nextinstruction. Consider here that, for example, a message “CLOSE THE DOOR”is displayed (Step SB4). When the worker closes the door 2006 inresponse to the instruction (Step SB5), the angular velocity sensor 2022attached to the door 2006 detects a corresponding signal.

In other words, when the door 2006 turns in a direction in which thedoor is closed, the angular velocity sensor 2022 detects the start ofthe turn. When the detection signal is transmitted, the edge device 42detects the “start of turn of door 2006”.

When the door 2006 is closed and its turn is stopped, the angularvelocity sensor 2022 detects the stoppage of the door 2006 (detects thatthe door is closed). If a sensor detection signal is transmitted at thistime, the edge device 42 determines that “the door is completely closed”(Step SB5). Then, the next instruction is transmitted to theglasses-type wearable terminal 1100. The edge device 42 transmits amessage such as “DOOR IS COMPLETELY CLOSED. PLEASE WAIT FOR A WHILE”(Step SB6).

The rotation axis direction serving as a criterion to detect therotation by the angular velocity sensors 2021 and 2022 does not alwayscoincide with the gravitation direction. For example, when the angularvelocity sensors 2021 and 2022 are placed in an oblique direction withrespect to the gravitation direction, the vertical-direction vibrationof the floor due to the movement of the worker is transmitted to theangular velocity sensors 2021 and 2022, and it is very likely thatdisturbance noise will be mixed into a rotation angle detection signal.

In order to reduce disturbance noise from the rotation angle detectionsignal, the three-axis acceleration sensor 72 is also built in at oncein the system (signal processing system) according to the presentembodiment. Accordingly, the direction θyz of the criterion B(gravitation direction) 104, which influences the angular velocitysensors 2021 and 2022, is extracted. If the signal processing (rotationtransformation) between the angular velocities is performed using theangle θyz, an angular velocity detection signal from which disturbancenoise components based upon an external vibration are reduced can begenerated. This signal processing brings about the advantage that theedge device 42 can correctly grasp the work (behavior) of the worker andthe conditions of the door 2006 and the screw 2001.

FIG. 41 shows a detailed configuration of the sensor device. The sensors2021 and 2022, which detect whether the worker has completed apredetermined work on a production site or the like, has a configurationcapable of additionally being set in the existing device (correspondingto the screw 2001 or door 2006 shown in FIG. 40 ) such as the existingenvironment or production facilities.

As a method for automatically detecting that a worker has completed hisor her work, there is a method for purchasing a new production devicethat incorporates in advance a plurality of sensors 2021 and 2022 fordetecting whether a predetermine work has been completed in order toreplace the existing device. This method however involves huge businessinvestment costs for the purchase. Adopting a method for additionallysetting the sensors 2021 and 2022, each of which is very inexpensive, inthe existing environment or production facilities brings about theadvantage capable of automatically detecting whether a worker hascompleted his or her work very inexpensively.

As a method for setting the sensors 2021 and 2022 automatically, abonding section 3008 is formed in a section where the sensors 2021 and2022 and the existing environment or production facilities are incontact with each other in the embodiment shown in FIG. 41 .Specifically, the bonding section 3008 for the contact portion of theexisting environment and the existing device can be formed of, e.g., anadhesive sheet whose strength is high. In this case, when the sensors2021 and 2022 are shipped, a cover sheet is attached in advance to thecontact portion of the existing environment and the existing device ofthe bonding section 3008. The cover sheet is removed in the place wherethe sensors 2021 and 2022 are placed, and the bonding section 3008 isbonded directly to the existing environment and the existing device. Thepresent embodiment is not limited to this. The bonding section 3008 forthe contact portion of the existing environment and the existing deviceis not caused to have any adhesive properties (or bonding properties) inadvance, but an adhesive can be caused to penetrate a portion in whichthe bonding section 3008 is in direct contact with the existingenvironment or the existing device and can be fixed thereto when thesensors 2021 and 2022 are placed. As another method for placing thesensors 2021 and 2022 additionally, they can be fixed to the existingenvironment or the existing device by a screw using the bonding section3008 for the contact portion of the existing environment and theexisting device.

In the configuration shown in FIG. 41 , an acceleration sensor unit oran angular velocity sensor unit 3006 is disposed adjacent to the bondingsection 3008 for the contact portion of the existing environment and theexisting device. The closer to the existing environment of a target tobe set additionally and the surface of the existing device the placewhere the acceleration sensor unit or the angular velocity sensor unit3006 is disposed, the more accurately the acceleration and angularvelocity of the existing environment and the existing device themselvescan be detected.

As shown in FIG. 41 , therefore, the acceleration sensor unit or theangular velocity sensor unit 3006 is disposed in a position closer to adevice (or an environmental object) to be set additionally than acontrol unit 3002, a short-range wireless communication unit 3004 or anenvironmental vibration generation device 3000. This brings about theadvantage that the acceleration and angular velocity of a subject(corresponding to the screw 2001 or door 2006 in FIG. 40 ) can bedetected.

As an acceleration sensor in the present embodiment, a low-Gacceleration sensor whose measurement range is not higher than 20 G (1 Grepresents the gravitational acceleration of the earth) is used. Whenthe low-G acceleration sensor is used as an acceleration sensor, theexterior wall section of the acceleration sensor unit or the angularvelocity sensor unit 3006 configures a fixed section and a sensor devicemovable section is placed therein (in the interior of the accelerationsensor unit or the angular velocity sensor unit 3006) (the detailedconfiguration of the interior is omitted from FIG. 41 ). Theacceleration is detected using variations in the position of the sensordevice movable section with the fixed section. In the presentembodiment, a capacitance detection system (which detects a variation incapacitance between the fixed section and the sensor device movablesection) or a piezoresistive system (which detects a distortion causedin a spring portion connecting the fixed section and the sensor devicemovable section using a piezoresistive element disposed in the springportion) can be used.

In the present embodiment, furthermore, a vibration system using amicro-electromechanical system (MEMS) can be utilized as an angularvelocity sensor (gyro sensor). Like the foregoing acceleration sensor,the angular velocity sensor (gyro sensor) is basically configured by afixed section that is configured by the exterior wall of theacceleration sensor unit or the angular velocity sensor unit 3006 and asensor device movable section placed therein (in the interior of theacceleration sensor unit or the angular velocity sensor unit 3006). Inthe fixed section, first comb-teeth electrodes and second comb-teethelectrodes are arranged orthogonally. A voltage is applied to the firstcomb-teeth electrodes alternately to vibrate the sensor device movablesection at fixed periods. When the acceleration sensor unit or theangular velocity sensor unit 3006 rotates, Coriolis force is generatedand the sensor device movable section rotates relative to the fixedsection. Then, the angular velocity is detected by considering therotational displacement as a variation in capacitance using the secondcomb-teeth electrodes. The present embodiment is not limited to theforegoing mechanical system, but a magnetism system, an optical systemor a mechanical system can be used in the angular velocity sensor (gyrosensor).

The data based upon the acceleration and angular velocity detected bythe foregoing method is transmitted to the edge device 42 via theshort-range wireless communication unit 3004. The control of theoperation of the short-range wireless communication unit 3004 and theprocessing of signals obtained from the acceleration sensor unit or theangular velocity sensor unit 3006 are performed by the control unit3002. As shown in FIG. 41 , the short-range wireless communication unit3004 and the control unit 3002 are arranged in the same row, whichbrings about the advantage capable of decreasing the height of thesensors 2021 and 2022 themselves.

As shown in FIG. 41 , in the present embodiment, the environmentalvibration generation device (a piezoelectric system or an electrostaticsystem) 3000 supplies power (power source) necessary for operating theacceleration sensor unit or the angular velocity sensor unit 3006, andthe short-range wireless communication unit 3004 and the control unit3002. The use of wires as a power source (power supply) to the sensors2021 and 2022 involves complication associated with a change in wiringmade each time the positions of the sensors 2021 and 2022 are changed.If a number of sensors 2021 and 2022 are placed using a replaceablebattery as a power source (power supply), the problem that the batteryreplacement operation is very complicated will occur. In the presentembodiment, the feature that the sensors 2021 and 2022 detectacceleration and angular velocity is used and the energy of theacceleration and angular velocity to be detected is used as a powersource (power supply) is used. It is thus unnecessary to supply power bywire, which brings about the advantage that a worker is relieved of notonly complexity of wiring change due to a change in placement positionof the sensors 2021 and 2022 but also complexity of battery exchange.

In the event of an earthquake, the upper floor of a high-rise buildingis generally shaken more than a one-story building. In such a structureprojecting from the shaking surface, a greater shake occurs in alocation away from the direct shaking surface (shaking amplitude islarge). Using this phenomenon, in the present embodiment, as shown inFIG. 41 , the environmental vibration generation device 3000 is disposedat a position farthest from the bonding section 3008 for the contactportion of the existing environment and the existing device. In otherwords, the environmental vibration generation device 3000 is disposed ata position farther from the bonding section 3008 for the contact portionof the existing environment and the existing device than theacceleration sensor unit or the angular velocity sensor unit 3006 or theshort-range wireless communication unit 3004 or the control unit 3002.Accordingly, the advantage capable of maximizing power generationefficiency is brought about.

FIG. 42 shows a basic configuration of the environmental vibrationgeneration device 3000. Part of the configuration is similar to thebasic configuration of the acceleration sensor or the angular velocitysensor described above. In other words, the environmental vibrationgeneration device 3000 includes a fixed section 3100 and a sensor devicemovable section 3102, and the sensor device movable section 3102 ismovable with respect to the fixed section 3100 in response to theexternal environmental vibration.

An instantaneous voltage generation unit 3104 is so formed that it canbe moved in synchronization with the movement of the sensor devicemovable section 3102 to generate an instantaneous voltage in accordancewith the movement of the sensor device movable section 3102. To use apiezo (piezoelectric) device as the instantaneous voltage generationunit 3104 is referred to as “a piezoelectric system” and to use anelectret (an insulator with semi-permanent charges) is referred to as“an electrostatic system”.

The instantaneous voltage generated by the instantaneous voltagegeneration unit 3104 is boosted along with DC electrification andsmoothing in a voltage boosting unit 3106. Then, the power output fromthe voltage boosting unit 3106 is stored in a storage unit 3108.

A specific operation principle of the environmental vibration generationdevice 3000 shown in FIG. 42 will be described with reference to FIGS.43 to 47 . In both cases where the “piezoelectric system” is adopted andthe “electrostatic system” is adopted for the instantaneous voltagegeneration unit 3104, the operations subsequent to that of the voltageboosting unit 3106 are common to both the cases as shown in FIGS. 43 to47 . In the case of the “piezoelectric system”, the output of apiezoelectric device 3130 is connected to an input terminal 3116. In thecase of the “electrostatic system”, the output of a metal electrodesubstrate 3138 is connected to the input terminal 3116.

Specifically, as shown in FIGS. 43 to 47 , in the “piezoelectricsystem”, a connecting section that connects the fixed section 3100 andthe sensor device movable section 3102 corresponds to the instantaneousvoltage generation unit 3104, and the piezoelectric device 3130 isdisposed in the connecting section. If the sensor device movable section3102 is greatly displaced from a neutral position with respect to thefixed section 3100, an electromotive voltage is generated between bothterminals of the piezoelectric device 3130. If the sensor device movablesection 3102 returns to the neutral position, an electromotive voltageto be generated between both terminals of the piezoelectric device 3130is reduced.

As shown in FIGS. 43 to 47 , an electret material 3134 is set in thefixed section 3100 in the “electrostatic system”. The electret means aninsulator with semi-permanent charges. As a specific material thereof,for example, CYTOP (registered trademark) can be used. In the embodimentshown in FIGS. 43 to 47 , the surface of the electret material 3134 isalways charged negatively. An electret electrode substrate 3132 isconnected to the electret material 3134 to hold the relative potentialof the electret material 3134 constantly at “0 V”. An opposing electrode3136 in a movable state is disposed close to the electret material 3134that is charged negatively. As the opposing electrode 3136 moves to theelectret material 3134, an instantaneous voltage is generated. Thus, theopposing electrode 3136 is disposed in the instantaneous voltagegeneration unit 3104 described with reference to FIG. 42 . The metalelectrode substrate 3138 is connected to the opposing electrode 3136,and the opposing electrode 3136 is supplied with charges through themetal electrode substrate 3138. Thus, the sensor device movable section3102 described with reference to FIG. 42 includes the metal electrodesubstrate 3138. It can be said that the sensor device movable section3102 or the instantaneous voltage generation unit 3104 can be configuredby the combination of the metal electrode substrate 3138 and theopposing electrode 3136. Based upon the electromagnetic capacitortheory, the absolute value of the amount of negative charges on thesurface of the electret material 3134 and the amount of positive chargeson the opposing surface of the opposing electrode 3136 close theretoneed to coincide with each other. If, therefore, the position of theopposing electrode 3136 coincides with that of the electret material3134, the largest amount of positive charges gather on the opposingsurface of the opposing electrode 3136. If the position of the opposingelectrode 3136 is greatly displaced from that of the electret material3134, the amount of positive charges gathering on the opposing surfaceof the opposing electrode 3136 becomes small. The positive chargesgathering on the opposing surface move to another place through themetal electrode substrate 3138.

Though not shown in FIG. 42 , FIGS. 43 to 47 show that a signaldetection unit 3110 is disposed at the exit of the instantaneous voltagegeneration unit 3104. Using the output from the signal detection unit3110, acceleration and angular velocity can be detected. Specifically, aresistor 3120 is set in the signal detection unit 3110, andinstantaneous current generated from the instantaneous voltagegeneration unit 3104 flows through the resistor 3120. When current flowsthrough the resistor 3120, a voltage is generated instantaneously ateither end of the resistor 3120. The instantaneous voltage is bufferedby a differential buffer amplifier 3112 to allow a change ininstantaneous current to be monitored from outside.

FIGS. 43 to 47 show the use of a Cockcroft-Walton circuit as an exampleof the voltage boosting circuit 3106. The present embodiment is notlimited to this circuit, but another circuit can be used if it is ableto at least rectify and smoothen a current or amplify a voltage.Furthermore, there is a capacitor element 3128 as an example of thestorage unit 3108. However, the present embodiment is not limited to thecapacitor element, but another rechargeable storage battery can be used.

Arrow 3114 shown in FIGS. 43 to 47 indicates a direction in which thesensor device movable section moves, and arrow 3142 indicates adirection of current. As shown in FIG. 43 , when the sensor devicemovable section 3102 moves to the left side, the amount of distortion ofthe piezoelectric device 3130 becomes small and thus an electromotivevoltage between both terminals (both surfaces) in the piezoelectricdevice 3130 becomes low. Accordingly, the reduced positive charges flowfrom the input terminal 311 to the piezoelectric device 3130. In thisdescriptions the right side and left side represent the directions onthe drawings.

In the electrostatic system, when the position of the opposing electrode3136 is moved to the left side, the amount of positive charges depositedon the surface of the opposing electrode 3136 increases and thus thedeposited positive charges flow into the input terminal 3116 through themetal electrode substrate 3138. As a result, even in both cases of the“piezoelectric system” and the “electrostatic system”, current 3184flows from the right side to the left side in the resistor 3120. Sincethe positive charges are supplied from the left-side electrode of acapacitor element 3122-1, the left-side electrode is charged positivelyafter the supply. Then, based upon the electromagnetic capacitor theory,the current 3148 flows into the right-side electrode of the capacitorelement 3122-1 through a diode element 3126-1 to supply the positivecharges. As another example, when the sensor device movable section 3102moves to the left side, if no charges remain at both electrodes of thecapacitor element 3122-1, the potentials of the electrodes becomenegative. Thus, it can also be said that the current 3184 flows towardthe right-side electrode of the capacitor element 3122-1.

When the sensor device movable section 3102 moves to the right side asshown in FIG. 44 , the electromotive power increases at either end ofthe piezoelectric device 3130 and current flows from the left side tothe right side in the resistor 3120 in the “piezoelectric system”. Inthe “electrostatic system”, the position of the opposing electrode 3136is greatly displaced from that of the electret material 3134 and thuscurrent flows from the left side to the right side in the resistor 3120in order to reduce the amount of positive charges deposited on thesurface of the opposing electrode 3136. At this time, the positivecharges stored in the right-side electrode of the capacitor element3122-1 move to the right-side electrode of a capacitor element 3122-2through a diode element 3126-2. Then, negative charges are stored in theleft-side electrode of the capacitor element 3122-2 to cancel thepositive charges. As for this phenomenon, the following description canalso be given. When the sensor device movable section 3102 moves to theright side, the potential of the right end of the resistor 3120 becomespositive. In the charge distribution in the capacitor element 3122-1shown in FIG. 43 , the potential of the right-side electrode of thecapacitor element 3122-1 becomes very high and thus the current 3148flows through the diode element 3126-2, Consequently, positive chargesare stored in the right-side electrode of the capacitor element 3122-2and negative charges are stored in the left-side electrode thereof.

After that, when the sensor device movable section 3102 returns to theleft side as shown in FIG. 45 , the current 3148 flows from the rightside of the resistor 3120 to the left side thereof. If the chargedistribution in either electrode of the capacitor element 3122-1 remainsunchanged as shown in FIG. 44 , the potential of the right-sideelectrode of the capacitor element 3122-1 becomes very low. As a result,the current 3148 flows toward the right-side electrode of the capacitorelement 3122-1 through the diode element 3126-1 and positive charges arestored in the right-side electrode of the capacitor element 3122-1. Atthe same time, current flows into the instantaneous voltage generationunit 3104 from the left-side electrode of the capacitor element 3122-1through the resistor 3120, with the result that negative charges arestored in the left-side electrode of the capacitor element 3122-1.

FIG. 46 shows the sensor device movable section that has started to movetoward the right side. It shows a momentary state in which the chargedistribution in the electrodes at both ends of the capacitor element3122-1 is held as shown in FIG. 45 when the current 3148 starts to flowfrom the left side of the resistor 3120 to the right side thereof at thestart of the movement. Since, in this case, the potential of theright-side electrode of the capacitor element 3122 becomes very high,current starts to flow from the right-side electrode of the capacitorelement 3122-1 to the right-side electrode of a capacitor element 3122-3through diode elements 3126-2 and 3126-3. As a result, the chargedistribution is generated in the electrodes at both ends of thecapacitor element 3122-3 (voltages are generated/held at both ends ofthe capacitor element 3122-3) as shown in FIG. 47 . The voltages arethus stored sequentially at both ends of the capacitor elements 3122-2to 3122-8.

In the present embodiment, the interior of the sensor shown in FIG. 41is so configured that the acceleration sensor unit or the angularvelocity sensor unit 3006 and the environmental vibration generationdevice 3000 are disposed separately from each other. In contrast tothis, as an application example of the present embodiment, both can beformed integrally as one unit. The basic structure in this case is shownin FIG. 48 . If a section that generates an acceleration signal or anangular velocity signal and an environmental vibration generationsection are formed integrally as one unit, the advantage capable ofdecreasing the total size of the sensors 2021 and 2022 is brought about.

In FIG. 48 , the fixed section 3100 includes a plurality ofinstantaneous voltage generation units (1) 3104-1 to (n) 3104-n. Each ofthe instantaneous voltage generation units (1) 3104-1 to (n) 3104-nincludes signal detection units (1) 3110-1 to (n) 3110-n. The detailedconfiguration of each of the instantaneous voltage generation units (1)3104-1 to (n) 3104-n and that of each of the signal detection units (1)3110-1 to (n) 3110-n may be the same as that of the instantaneousvoltage generation unit 3104 and that of the signal detection unit 3110described with reference to FIGS. 43 to 47 . The present embodiment isnot limited to this configuration, but another configuration can beemployed if it has a function of fulfilling a similar function. Sincethe fixed section 3100 is disposed in common to the instantaneousvoltage generation units (1) 3104-1 to (n) 3104-n (the fixed section3100 is shared with the instantaneous voltage generation units, theadvantage capable of decreasing the total size of the sensors 2021 and2022 is brought about.

A signal computation unit 3200 computes a signal detected from each ofthe signal detection units (1) 3110-1 to (n) 3110-n to extract anacceleration signal and an angular velocity signal.

In parallel with the foregoing signal processing circuit, voltageboosting units (1) 3106-1 to (n) 3106-n are disposed to correspond tothe instantaneous voltage generation units (1) 3104-1 to (n) 3104-n,respectively. The detailed configuration of the voltage boosting units(1) 3106-1 to (n) 3106-n may be the same as that of the voltage boostingunit 3106 described with reference to FIGS. 43 to 47 . The presentembodiment is not limited to this configuration, but anotherconfiguration can be employed if it has a function of fulfilling asimilar function. The outputs of the voltage boosting units (1) 3106-1to (n) 3106-n are synthesized by a synthesis unit 3210 and thenconnected to the storage unit 3108. In FIG. 48 , the synthesis unit 3210is configured to be connected electrically. As shown in FIGS. 43 to 47 ,the capacitor element 3128 is disposed immediately before the exit ofeach of the voltage boosting units (1) 3106-1 to (n) 3106-n in order toprevent a backflow and thus no problems occur though it is simplyelectrically connected as shown in FIG. 48 . The present embodiment isnot limited to this configuration. The powers can be synthesized by asophisticated method.

As a specific example of arrangement of the instantaneous voltagegeneration units (1) 3104-1 to (n) 3104-n shown in FIG. 48 , anembodiment using, e.g., the “electrostatic system” will be described.FIG. 49 shows a one-way direction section arrangement. In the commonfixed section 3100, electret electrode substrates (1) 3132-1 to (3)3132-3 and electret materials (1) 3132-1 to (3) 3132-3 are laminated oneon another.

In the center of the movable section, a triangular prism-shaped movablesupport unit 3210 is disposed movable relative to the fixed section3100. The movable support unit 3210 can be moved in a directionorthogonal to the drawing of FIG. 49 (front-side direction and back-sidedirection). On the sides (rectangular surfaces) of the triangular prismof the movable support unit 3210, metal electrode substrates (1) 3138-1to (3) 3138-3 and opposing electrodes (1) 3136-1 to (3) 3136-3 aredisposed and all of them are synchronized and can be moved.

FIG. 50 shows a relationship in arrangement among the opposingelectrodes (1) 3136-1 to (3) 3136-3 along the moving direction of themovable support unit 3210. The opposing electrodes (1) 3136-1 to (3)3136-3 corresponding to the electret materials (1) 3134-1 to (3) 3134-3are displaced from one another. This displacement brings about theadvantage capable of detecting not only the absolute values of anacceleration amount and angular velocity but also the directions thereofsimultaneously.

Consider the case where the metal electrode substrates (1) 3138-1 to (3)3138-3 are simultaneously displaced horizontally based on therelationship in position shown in FIG. 50 . In this case, the absolutevalue of negative charges deposited on the surface of the opposingelectrode (2) 3136-2 decreases irrespective of the displacementdirection. If the metal electrode substrates (1) 3138-1 to (3) 3138-3are displaced at once to the right side, the absolute value of negativecharges deposited on the surface of the opposing electrode (1) 3136-1does not vary, but the absolute value of negative charges deposited onthe surface of the opposing electrode (3) 3136-3 increases. If the metalelectrode substrates (1) 3138-1 to (3) 3138-3 are displaced at once tothe left side to the contrary, the absolute value of negative chargesdeposited on the surface of the opposing electrode (3) 3136-3 does notvary, but the absolute value of negative charges deposited on thesurface of the opposing electrode (1) 3136-1 increases. From thestrength and direction of current flowing toward the opposing electrodes(1) 3136-1 to (3) 3136-3 (the results of signal computation in thesignal computation unit 3200), the moving direction and the moving speedvariation of the metal electrode substrates (1) 3138-1 to (3) 3138-3 canbe seen.

The present embodiment is not limited to the arrangement shown in FIG.50 but can be directed to another arrangement. For example, the opposingelectrodes (1) 3136-1 to (3) 3136-3 can be aligned with one another andinstead the electret materials (1) 3134-1 to (3) 3134-3 can be displacedfrom one another.

The one-axis direction movement of the movable support unit 3210 hasbeen described above. If the same principle is expanded, three-axisdirection acceleration and three-axis angular velocity can be detected.

As is seen from the description given with reference to FIGS. 43 to 47 ,the environmental vibration generation device 3000 shown in FIG. 41generates acceleration or angular velocity continuously to storevoltages in the capacitor elements 3122-1 to 3122-8 gradually.Conversely, when neither acceleration nor angular velocity is generatedfor a long period of time, the power stored in the storage unit 3108 (inthe capacitor element 3124 in the interior thereof (FIGS. 43 to 47 )) isgradually discharged. Thus, when the environmental vibration generationdevice 3000 is in a stationary state for a long period of time, itbecomes difficult to supply drive power to the acceleration sensor unitor the angular velocity sensor unit 3006, the short-range wirelesscommunication unit 3004 and the control unit 3002. In the presentembodiment utilizing this feature, acceleration or angular velocity isoutput with timing when it becomes low. This brings about the advantagecapable of securing a stable supply of power from the environmentalvibration generation device 3000 and detecting the changed accelerationor angular velocity with high accuracy.

While a worker is working, the sensors 2021 and 2022 is vibrating androtating. Power of the environmental vibration generation device 3000 isthus stored in the interior of the sensors 2021 and 2022. When theworker's working is completed, the vibration and rotation of the sensors2021 and 2022 is stopped, with the result that the edge device 42 isinformed of the stoppage of vibration and rotation of the sensors 2021and 2022 during a period in which the power storage amount of theenvironmental vibration generation device 3000 is secured.

The timing of change of the acceleration or angular velocity can beextracted by the control unit 3002 shown in FIG. 41 and so can be anacceleration value or an angular velocity value immediately after theextraction of the timing. A method for extracting the timing of changeof the acceleration or angular velocity and an acceleration value or anangular velocity value immediately after the extraction of the timingwill be described with reference to FIG. 51 . The values of theacceleration or angular velocity obtained from the signal computationunit 3200 shown in FIG. 48 are input to the control unit 3002. Thecontrol unit 3002 includes a reference timing generation unit 3302 andprocesses an acceleration signal or an angular velocity signaltransmitted from the signal computation unit 3200 each time referencetiming is generated from the reference timing generation unit 3302.

As an index for detecting a variation in acceleration or angularvelocity, in the angular velocity, an accumulated value of “angularvelocity in a constant rotation direction” or an average value for eachtiming described above can be used. In the acceleration, the inversionof the acceleration direction is often repeated; thus, “an absolutevalue of acceleration” or “an amplitude value of a signal varying in thepositive and negative directions” or the like can be calculated tocompute an accumulated value or an average value for each timingdescribed above. The present embodiment is not limited to this. Anabsolute value is computed and an amplitude value is calculated for theangular velocity, and an accumulated value and an average value can becalculated for the acceleration in consideration of the direction. Thesecomputations are performed by a calculation unit 3304 for calculating apower storage amount or an average value within a given period of time.

In the present embodiment, a calculated value and its subsequentcalculated value are compared every predetermined timing in order toextract timing of change. In other words, the index obtained from thecalculation unit 3304 for calculating a power storage amount or anaverage value within a given period of time, is stored temporarily in acalculation result temporal storage unit 3306 and compared with anindex, which is obtained immediately after the former index from thecalculation unit 3304, by a comparison unit 3308. When a result of thecomparison exceeds a preset given value (including both the case wherean index value is larger than the given value and the case where it issmaller than the given value), the comparison unit 3308 considers thatthe index value “has greatly varied” and outputs a voltage to (displaysa flag on) a change timing notification terminal 3314. Thus, a point ofswitching between output values of the change timing notificationterminal 3314 represents change timing. Simultaneously, an index valueimmediately after the change is output to a changed-value outputterminal 3312.

In FIG. 51 , the control unit is shown in the form of a block diagram ofa circuit. The present embodiment is not limited to this. The foregoingprocessing method can be executed by programs/software to be executed bya processor.

Using the switching point timing of the output values of the changetiming notification terminal 3314 as a trigger, the output values of thechanged-value output terminal 3312 are communicated to the edge device42 from the short-range wireless communication unit 3004 (FIG. 41 ).

The sensors 2021 and 2022 capable of detecting acceleration or angularvelocity as described above can be used not only in the workplacedescribed with reference to FIG. 40 but also in any other applicationfields. For example, they can be used in the infrastructure healthmarket such as an automatic deterioration diagnosis of infrastructureunder social infrastructure environment. Specifically, the sensors 2021and 2022 used in the system according to the present embodiment can beused in a hammering test to inspect a partial deterioration of bridgesand tunnels (a deteriorated portion is predicted from the pitch and toneof sound generated by hammering part of the infrastructure). In thiscase, it is bonded to the pillars, walls and ceilings of a bridge and atunnel using the bonding section 3008 to be bonded to the existingenvironment and existing device. The vibration generated when a workerhammers a specific portion is sensed by the sensors 2021 and 2022 andthe results of the vibration are collected by the edge device 42, withthe result that a deterioration can be predicted.

In FIG. 52 , (a) shows steps before and after a screwing operation. Inother words, the vibration state before a worker approaches is a stateof a normal period 3402. When the worker starts to tighten a screw, thenormal period shifts to a screwing period 3404. when the screwing iscompleted, the screwing period shifts to a screwing completion period3406.

In FIG. 52 , (b) shows a measured acceleration value or a measuredangular velocity value in the position of the screw 2001 in each step.In the normal period 3402 prior to the screwing operation, the vibrationstate is a normal state. In the screwing completion period 3406, too,the vibration state returns to the normal vibration state. As a result,the acceleration or angular velocity greatly changes at the instant whenthe normal period 3402 shifts to the screwing period 3404 and at theinstant when the screwing period 3404 changes to the screwing completionperiod 3406.

The instant when the acceleration or angular velocity has greatlychanged is extracted automatically and immediately afterward anacceleration value or an angular velocity value (or an amount ofaccumulation or an average during a predetermined period) is transmittedto the edge device 42 as the information described with reference to (c)of FIG. 52 .

FIG. 53 shows a variation in angular velocity that is detected by thesensor 2022 attached to the door 2006 in the embodiment described withreference to FIG. 40 when the door 2006 is closed. As shown in (a) ofFIG. 53 , the timing can be divided into a time period during which thedoor is stopped 3502, a time period during which the door is turned 3504and a time period during which the door is closed 3506. In FIG. 53 , (b)shows a variation in angular velocity that is detected by the sensor2022 attached to the door during each of the time periods. The value ofangular velocity increases during the time period during which the dooris turned 3504 and becomes the largest immediately before the door isclosed. An example of the amount of power storage (amount of powergeneration) in the environmental vibration generation device 3000 (FIG.41 ) is shown in (c) of FIG. 53 . The environmental vibration generationdevice 3000 does not start to generate power (store power) before thedoor starts to turn. Only during the time period in which the amount ofpower storage exceeds a predetermined value corresponds to an operationperiod 3508 of the short-range wireless communication unit 3004 and thecontrol unit 3002.

Short-range wireless communication can be carried out only during theoperation period 3508. As shown in (e) of FIG. 53 , therefore,communication information is transmitted to the edge device 42 laterthan the start of turn of the door. The point at which the time periodduring which the door is turned 3504 is switched to the time periodduring which the door is closed 3506 is included in the operation period3508, information indicating that “the angular velocity of the door iszero” is communicated immediately after that point.

In the system according to the embodiment shown in FIG. 39A or 39B, whenpower is always stably supplied to sensors 2500 and 1152, short-rangewireless communication can be carried out between the sensors 2500 and1152 and the edge device 42 at any time. Thus, the timing of short-rangewireless communication between the sensors 2500 and 1152 to which powercan stably be supplied or the glasses 1100 and the edge device 42 isbasically controlled by the edge device 42.

In contrast, the sensors 2021 and 2022 that receive power from theenvironmental vibration generation device 3000 is able to carry outshort-range wireless communication only during the operation period 3508as shown in (d) of FIG. 53 . This timing cannot be predicted by the edgedevice 42. In the system according to the present embodiment, therefore,only the sensors 2021 and 2022 that receive power from the environmentalvibration generation device 3000 is provided with the initiative oftiming of the short-range wireless communication. This brings about theadvantage capable of stable short-range wireless communication.

In this case, the timing of short-range wireless communication managedby the edge device 42 and the timing of short-range wirelesscommunication performed voluntarily by the sensors 2021 and 2022 overlapeach other to cause an unstable factor of short-range wirelesscommunication. To resolve this problem, in the system according to thepresent embodiment, a wireless band (wireless reference frequency) ofshort-range wireless communication managed by the edge device 42 and awireless band (wireless reference frequency) of short-range wirelesscommunication performed voluntarily by the sensors 2021 and 2022 areseparated from each other to prevent a cross talk therebetween. Thus,the short-range wireless communication managed by the edge device 42 canbe stabilized.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the inventions. Indeed, the novel embodiments described hereinmay be embodied in a variety of other forms; furthermore, variousomissions, substitutions and changes in the form of the embodimentsdescribed herein may be made without departing from the spirit of theinventions. The accompanying claims and their equivalents are intendedto cover such forms or modifications as would fall within the scope andspirit of the inventions.

What is claimed is:
 1. A behavior estimating method for estimatingbehavior of a person to be measured equipped with an activity meterincluding a signal processing unit that includes a battery, a three-axisacceleration sensor, a three-axis geomagnetism sensor, a signalamplifier group, an A/D converter, a controller, a memory, acommunication controller, and a behavior estimation engine, the methodcomprising: amplifying three-axis-direction detected raw signalsobtained from the three-axis acceleration sensor and the three-axisgeomagnetism sensor; converting the three-axis-direction detected rawsignals into digital signals; storing the digital signals in the memory;signal processing using the digital signals by the controller; whereinthe signal processing includes: presetting a first angle between agravitation direction and a direction of action when a first person withan average height handles an object to be operated; calculating thedigital signals acquiring different acceleration waveforms, the digitalsignals being acquired when a second person being measured handles theobject; extracting a second angle between the gravitation direction andthe direction of action of the second person being measured from thedifferent acceleration waveforms, wherein the gravitation direction isobtained instantaneously; performing a coordinate transformation processamong the different acceleration waveforms based upon a difference ofthe preset first angle and the extracted second angle; sending a resultof the coordinate transformation to the behavior estimation engine; andperforming behavior estimation of the second person being measured usingthe result.
 2. The behavior estimating method of claim 1, wherein athreshold value used for the behavior estimation is allowed to bechanged depending upon the second angle.
 3. The behavior estimatingmethod of claim 1, wherein an extension/contraction matching techniqueis employed in at least part of the behavior estimation process.
 4. Abehavior estimating system comprising: a battery; a three-axisacceleration sensor; a three-axis geomagnetism sensor; a signalamplifier group; an A/D converter; a controller; a memory; acommunication controller; and a behavior estimation engine; wherein: thethree-axis acceleration sensor and the three-axis geomagnetism sensorgenerate three-axis-direction detected raw signals; the signal amplifiergroup amplifies the three-axis-direction detected raw signals; the A/Dconverter converts the three-axis-direction detected raw signals intodigital signals; the memory stores the digital signals; the controllerprocesses the digital signals; a first angle between a gravitationdirection and a direction of action of a first person with an averageheight is preset when the first person handles an object to be operated;a second angle between the gravitation direction and the direction ofaction of a second person being measured is obtained when the secondperson handles the object, wherein the gravitation direction is obtainedinstantaneously; a difference between the first angle and the secondangle is obtained, and the gravitation direction and the direction ofaction of the second person are coordinate transformed based on thedifference of the first angle and the second angle; the result data ofthe coordinate transformation is sent to the behavior estimation engine;and behavior estimation of the second person is measured using theresult data.
 5. The behaver estimation system of claim 4, wherein thebehavior estimation engine is provided in an edge device.
 6. The behaverestimation system of claim 4, wherein the battery, the three-axisacceleration sensor, the three-axis geomagnetism sensor, the signalamplifier group, the A/D converter, the controller, the memory, and thecommunication controller are provided in a wrist band.